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EXECUTIVE SUMMARY 



In 1992, the U.S. Department of Education (ED) awarded 3 -year grants to five 
states and the District of Columbia for the development of curriculum frameworks in 
mathematics or science for grades K-12, together with new approaches to teacher 
education, certification, and professional development. These grants averaged about 
$850,000 and ran through September 30, 1995. In 1993, ED awarded 3-year grants to 10 
additional states. 

As part of a federal strategy to promote systemic, standards-based reform, 
curriculum frameworks were intended to play a central role in translating the national 
standards to state policy-makers, district officials, and local educators. In the original 
solicitation, curriculum frameworks were defined as “guidelines for the content of the 
curriculum and for how that content should be organized.” The projects were also 
charged with the development of “other products,” specifically: 

• Develop model guidelines for effective approaches to teacher education 
and certification based upon world class standards and the State 
curriculum frameworks tied to those standards; and 

• Develop criteria for teacher recertification, and design and pilot test a 
model, cost-effective inservice professional development program for 
teachers based upon world class standards and the State curriculum 
frameworks tied to those standards. [Federal Register, 57(146), July 29, 

1992, pp. 33603-33604] 

This final report on the Dwight D. Eisenhower State Curriculum Frameworks 
Projects uses the findings from the evaluation of the 16 projects to better understand the 
implementation of standards-based reform at the state and district levels. 

' ' ' ’ i‘ # V 

Progress of the Projects 

Although there was wide variation in the format and quality of the documents that 
the project states produced, 15 of the 16 states completed curriculum frameworks as a 
result of their grants. One state, Arizona, halted work on the project before completing a 
framework. Regardless of the progress of the projects, all 16 states that received 
curriculum framework grants were actively engaged in standards-based reform. Even in 
those states where the framework projects ran into difficulty, state officials and policy- 
makers were reviewing and revising policies and programs under the banner of standards. 
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However, the 16 projects devoted less time and resources to the project’s other 
products: model guidelines for teacher education and certification, criteria for teacher 
recertification, and model professional development. As a result, the projects were 
unable to meet the full expectations of the original solicitation. Those states that were 
best able to claim some progress did so as a result of activities already under way in the 
state. This progress can be summarized as follows: 

• Only Delaware, Michigan, New York, and Oregon formally designed, 
piloted, and evaluated model professional development programs, 
although most states did conduct framework-based professional 
development as an implementation strategy. 

• Eight of the framework projects claimed progress toward drafting 
guidelines for teacher education and certification. 

• Eight states reported changes in teacher recertification policies. 

Given that 48 states have developed or are developing standards documents in core 
discipline areas, it seems likely that the 1 6 states that received Eisenhower State 
Curriculum Frameworks Project grants would have developed curriculum frameworks or 
other standards documents without the grants. However, the grants brought resources 
that enriched the framework development process. Federal funds allowed project states 
more resources and therefore more time, professional staffing, and opportunities for 
professional development than was the case in most non-project states. 

Even though the states’ progress on the other products was uneven, a case can be 
made that federal funds did help those states that tried to develop model professional 
development programs, model guidelines for teacher education and certification, and 
criteria for teacher recertification. However, on the face of it, the value added by this 
investment seems to be significantly lower than that of the investment in development of 
frameworks. 

The grants did not guarantee a high-quality or even a completed document or 
product, but every state official involved in the projects can point to a positive 
contribution of the grant. Despite similar ranges of quality among project and non- 
project states, project states, in general, produced more comprehensive documents. Just 
how lasting the contributions of the framework documents will be remains an open 
question. 



10 

er|c 



VI 



Developing Frameworks and Other Products: Strategies and Issues 

The typical framework project followed a development process that had the 
following characteristics: 

• The development process was typically organized through the state department 
of education. 

• Committees comprised of university scholars, discipline specialists, teachers, 
administrators, and occasionally business and community representatives were 
charged with developing the curriculum frameworks. 

• The actual writing of the document was done by a subcommittee, which brought 
successive drafts to the full committee for review and comment. 

• Typically, the development process began by reviewing national standards 
(NCTM, Benchmarks, and the National Science Education Standards), but the 
committees viewed these documents as sources of ideas and guides, not as 
something to be strictly followed. 

Framework development strategies varied among states, but all were affected by the 
political context, either in initial choices about the projects or by midcourse corrections 
necessary to develop documents that were relevant to local educators and acceptable to 
state leaders. The project states were careful to avoid public controversy, thereby 
creating a tension between consensus building and reform. Thus, the state departments 
(and state boards in some states) faced the challenge of promoting change while avoiding 
discord. This delicate balance required the use of language that the public understood in 
order to reduce the chances of a negative reaction, but risked getting no reaction, thereby 
weakening support for reform. One result was that standards and framework 
development became a professional, not a public, activity. However, obtaining public 
support for standards is likely to require varied educational opportunities, primarily at the 
local level, for the many publics to more fully understand standards and their 
implications. 

The projects used a variety of strategies in the development of model professional 
development programs, model guidelines for teacher education and certification, and 
criteria for teacher recertification. These included: 

• Drafting a document or a framework chapter that discusses or recommends 
particular courses of action in these areas of policy and practice. 

• Implementing a set of activities, usually framework-based workshops. 

• Working with or handing off to an existing task force or project the 
responsibility for addressing one of these issue areas. 




• Choosing not to address particularly difficult issues. 

The projects’ uneven progress on the other products was a result of the unrealistic 
assumptions of the original solicitation and the projects’ emphasis on framework 
development over work on the other products. In the future, ED might consider avoiding 
such multifaceted and complex awards. Alternatively, if ED targeted specific activities in 
multiple awards and required that the projects coordinate their efforts, or gave clear 
direction as to the level of effort expected on each facet of the grant, the states would be 
better able to complete all components of the project. 

Finally, the State Curriculum Frameworks Projects revealed a problem with ED’s 
financial management and monitoring systems. A new financial management system 
may improve the situation, but ED still faces the challenge of improving its monitoring 
system. 

Quality of the Curriculum Frameworks 

The study team convened a group of 1 8 distinguished mathematics and science 
educators. This group of reviewers developed a rubric for assessing the quality of 
framework documents and, using this rubric, assessed 12 framework documents. The 
major findings from this analysis are: 

• In general, the reviewers found that state frameworks showed marked progress 
in expanding beyond a basic-skills emphasis to focus more on higher-order 
skills for all students in mathematics and science. 

• At the same time, the reviewers found that some of the frameworks omitted 
some of the major categories of the national standards, suffered from a lack of 
usability, or failed to convey adequately how equity can be achieved. 

• Most frameworks presented sample activities and vignettes that often were 
either inconsistent with national standards or inadequately armotated and 
explained. 

• Frameworks tended to address classroom assessment, but not large-scale 
assessment. Many framework documents contained performance indicators for 
the classroom; however, few contained rubrics for evaluating levels of 
performance. 

Following the experts’ analysis, our study collected 10 local curriculum documents 
from 5 states and examined their alignment with the state frameworks. Some local 
documents were more explicitly aligned with the state frameworks than others. In a few 
cases, the local documents seemed not to reflect the state frameworks. High-quality state 



O 

ERIC 



viii i 2 



curriculum frameworks appear to help districts produce high-quality local curriculum 
frameworks or guides; however, they do not guarantee quality. 

Curriculum Frameworks and State Policy 

The State Curriculum Frameworks Projects corresponded with a period of active 
policy development in the states. Besides developing framework or standards documents, 
the states began developing and piloting new assessment systems, addressing the issue of 
teacher licensure, and reconsidering a host of other policies. States that received grants 
were already trying to implement reform, but in the vast majority of these states the 
framework projects were a resource for policy reform. Specifically: 

• All but one of the 16 project states were plaiming, developing, piloting, or 
implementing new statewide assessment systems. 

• In 10 of the 16 states, the projects’ frameworks played a central role in the 
assessment development process. 

• Nine of the 16 project states were developing new certification and/or new 
recertification requirements, had them under consideration by the appropriate 
policy-making body, or had them in place. 

• Six of those states used the frameworks in the development of new teacher 
licensure requirements. 

The glass is more than half full when it comes to the frameworks’ influence on state 
policy, although two factors tended to limit that influence: 

• As the name implies, the State Curriculum Frameworks Projects were viewed 
by some state officials as projects, just like hundreds of other projects they 
oversaw. 

• Even in states where project leadership thought of the project as part of a grand 
scheme for changing policy, there was no systematic plan for changing policy. 

Despite these limitations, the majority of frameworks developed through the projects 
were a useful resource in the states’ efforts to make education policy align with the goal 
of high standards for all students. 

Implementation Strategies for Improving Teacher Practice 

The State Curriculum Frameworks Projects employed a variety of strategies to use 
the frameworks to help improve teacher practice: 

• Thirteen of the 1 6 states used professional development and technical assistance 
to increase teacher understanding and use of the frameworks. 



• Six states produced resource guides or other support materials to supplement the 
frameworks. 

• Fourteen of the 16 states used technology to increase access and awareness of 
the frameworks. 

• Six states used pilot sites to serve as examples for other sites trying to use the 
frameworks. 

• Two states sponsored study groups for teachers to read and discuss the 
frameworks. 

Not all of these professional development opportunities were paid for with project 
funds, as the states made efforts to leverage resources to support the implementation 
process. The types of opportunities states offered varied across and within states on a 
number of dimensions, including content, intensity, and duration. 

The most ambitious projects maximized teachers’ exposure to the documents and 
provided both time and resources for teachers to study, reflect, and experiment. These 
strategies required a coordinated effort — one that garnered the resources of other state, 
federal, and local reform efforts. It also required that implementation strategies extend 
beyond the life of the grants. 

Impact in the Districts 

We visited 17 local districts in 8 states and highlight 5 cases in this report. Local 
districts used frameworks to revise their local curriculum guides and to improve teacher 
practice with varying degrees of success. However, the cases illustrate that professional 
practice can improve when teachers and others are involved in production of local 
standards. The cases also illustrate the conditions and circumstances that enhanced the 
effective use of frameworks and standards, including: 

• The best-case scenario schools and districts engaged the standards documents 
from a foundation of previous reform activity. 

• Framework use was most effective as part of a whole-school change strategy 
that promoted a collegial and professional school culture. 

• Extensive and intensive professional development opportunities that focused on 
standards were essential to using frameworks effectively. 

In addition to these supportive conditions, the cases revealed that districts and 
schools adapted the standards and framework documents rather than adopted them. 
However, in our best-case scenarios, the schools and the teachers had the capacity to add 
value to the guidance during the adaptation process. These cases suggest that reformers 
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at the top of the system are always dependent to some degree on reformers at the bottom 
of the system — suggesting that capacity building is a key to successful implementation. 

Even in districts that are using frameworks and standards effectively, some 
unresolved issues remain, including: 

• Local involvement with state frameworks tends to emphasize content over 
pedagogy. The imbalance between the need for changes in both what is taught 
and how it is taught stems, in part, from states’ reluctance to offer guidance on 
pedagogical issues. 

• In all the districts, teachers were struggling with the sometimes conflicting 
purposes of assessment: assessment for instruction vs. assessment for 
accountability. 

• The districts were only begirming to explore ways to build professional 
development into the structure and organization of the school day. 

As was evident in the district cases, standards-based reform has captured the 
attention of many professional educators who are trying to improve mathematics and 
science education, but another significant challenge remains. Even in the best-case 
scenario districts, standards-based reform has been primarily a professional, not a public, 
activity. This lack of public involvement with standards is a problem because, regardless 
of their effectiveness, reform leaders — from the superintendents to the principals — were 
vulnerable to shifts in public opinion and the political composition of the local school 
boards. Without stronger public support, even the most promising reform efforts could 
be scuttled. 

implications for Federal Efforts to Promote Standards-Based Reform 

Frameworks can serve as general policy guidance and can be useful to local districts 
in their reform efforts. Although federal funds lent support to standards-based reform in 
the states, each state’s political and educational context dictated what role the projects 
would play in the reform effort. The projects’ uneven efforts to develop model 
professional development programs, model guidelines for teacher education and 
certification, and criteria for teacher recertification also suggest a need for future ED 
grants to be based on a more realistic set of assumptions and a more reasonable scope of 
work. 

Much more work is needed before the curriculum frameworks will be well used in a 
majority of districts and schools. As our case study districts suggest, capacity building is 
a key to successful implementation. Districts and individual schools need more time and 



resources to translate the state frameworks into local curricular guidance, a process that 
can make the standards more meaningful. Federal support for local implementation of 
standards would build on the work of the State Curriculum Frameworks Projects and help 
expand the niunber of schools that are using standards to create fundamental changes in 
the way they work. 
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I. INTRODUCTION 



In 1992, the U.S. Department of Education (ED) awarded 3-year grants to the 
District of Columbia, Florida, Nebraska, New Jersey, New York, and Rhode Island for 
the development of curriculum frameworks in mathematics or science for grades K-12, 
together with new approaches to teacher education, certification, and professional 
development. These grants averaged about $850,000 and ran through September 30, 
1995. In 1993, 10 more awards were made to Alaska, Arizona, Arkansas, Delaware, 
Louisiana, Maine, Massachusetts, Michigan, Oregon, and Wisconsin. The second cohort 
of states were to complete their projects by September 30, 1996. This final report 
presents evaluation findings on the State Curriculum Frameworks (SCF) Projects and 
applies those findings to issues surrounding the implementation of standards-based 
reform at the state and district levels. 

This evaluation has been carried out as part of a larger study in which we are also 
examining the Regional Consortia Program, which, like the framework grants, is part of 
the U.S. Department of Education’s Dwight D. Eisenhower National Mathematics and 
Science Program. In this introductory section, we define the term curriculum framework 
and place the State Curriculum Frameworks Projects in the context of the current federal 
strategy for improving the nation’s schools. Following that discussion, we review the 
report’s purposes and methods of the study. Finally, we outline the remainder of the 
document. 

What Is a Curriculum Framework? 

Although there are a variety of definitions of a curriculum framework, there is 
general agreement that curriculum frameworks are based on standards. Standards 
establish what students should know and be able to do. In the case of the SCF Projects, 
standards for student learning were seen as national (although not federal) in character. 
Thus, the solicitation explicitly required projects to build on the standards developed by 
the National Council of Teachers of Mathematics (NCTM, 1989) and the emerging 
standards in science under development at that time by the American Association for the 
Advancement of Science (AAAS, 1993 — Benchmarks), the National Science Teachers 
Association (NSTA, 1992 — Scope, Sequence and Coordination), and/or the National 
Research Council (1996 — National Science Education Standards). 





In the original solicitation, curriculum frameworks were defined as “guidelines for 
the content of the curriculum and for how that content should be organized” and viewed 
as a bridge between the national standards and the classroom [Federal Register, 57(146), 
July 29, 1992, p. 33602]. The role of the states then was to communicate and adapt these 
national standards for their particular student populations. Given state-developed 
frameworks, local educators were “to implement or adapt them for themselves” [Federal 
Register, 57(146), July 29, 1992, p. 33602]. 

Typically, states refined the definition, making it clear that curriculum frameworks 
did not provide detailed curriculum or mandates for specific programs or methods. 
Instead, curriculum frameworks were general guides with many purposes, as this example 
from Massachusetts illustrates: 

Curriculum Frameworks are: 

• Guides for developing district-wide programs that demonstrate effective 
learning and teaching strategies; 

• Guides for creating detailed curriculum; 

• Guides for developing classroom and statewide assessments; 

• Guides for selecting instructional materials; 

• Guides for planning professional development; 

• « 

• Guides for restructuring schools to support learning; 

• Presentation of critical issues that underlie learning, teaching, and assessment; 
and 

• Documents that support teachers as they try innovations and investigations that 
provide high quality learning experiences for each and every student. (The . 
Massachusetts Curriculum Frameworks, Charting the Course: The Common 
Chapters of the Massachusetts Curriculum Frameworks, 1996, p. 6) 

Not all states had as ambitious agendas for their curriculum frameworks, but all states 
based their frameworks on the national standards and expected their frameworks to help 
advance standards-based reform. 

The State Curriculum Frameworks Projects in the Context of Systemic 
Standards-Based Reform 

The State Curriculum Frameworks Projects are an early example of the federal 
strategy to promote systemic, standards-based reform throughout the nation. The original 
solicitation makes this point quite clearly: “The Secretary takes this action to focus 



Federal financial assistance on State curriculum frameworks as the starting point for 
systemic improvement in mathematics and science education” [Federal Register, 

57(146), July 29, 1992, p. 33602]. Here, systemic reform refers to a model for improving 
schools that begins with the establishment of high standards for what all students should 
know and be able to do, the translation of these standards into concrete guidance for 
educators, and the alignment of the full range of policies and practices (e.g., assessment, 
preservice education, professional development) to support all students meeting those 
standards (Smith & O’Day, 1991). 

With the State Curriculum Frameworks Projects, the federal government was not 
attempting to support states in the full range of activities needed to undergo systemic 
standards-based reform. In fact, the original grant solicitation made it clear that the 
reform of certain components of the educational system, such as assessment, was beyond 
the scope of the SCF projects.' Yet, consistent with the model of reform, the projects 
were asked to do more than just develop framework documents. In particular, the 
solicitation called on grantees to: 

• Develop model guidelines for effective approaches to teacher education and 
certification based on world-class standards and the state curriculum 
frameworks tied to those standards. 

• Develop criteria for teacher recertification, and design and pilot test a model, 
cost-effective inservice professional development program for teachers based on 
world-class standards and the state curriculum frameworks tied to those 
standards. [Federal Register, 57(146), July 29, 1992, pp. 33603-33604] 

In summary, then, the SCF projects should be seen as an explicit federal effort to support 
systemic standards-based reform in the states. States, in turn, agreed to an ambitious 
reform agenda: first, developing curriculum frameworks in K-12 mathematics and/or 
science; second, designing new preservice education and certification model guidelines; 
third, developing criteria for teacher recertification; and fourth, devising and piloting new 
approaches to inservice professional development. Further, states took on these 
challenges in the context of an array of other educational reform efforts already under 
way, funded with both state and federal dollars. 



‘ The State Curriculum Frameworks Projects should be viewed within the context of the broader federal 
strategy. In particular, the subsequently enacted Goals 2000 legislation, the Improving America’s 
Schools Act (lASA), and the National Science Foundation’s Statewide Systemic Initiatives (SSI) 
program demonstrate further, and more comprehensive, federal support for systemic reform. 



Evaluation Purposes and Methods 



The overall purpose of the evaluation is to assess the degree to which the 
framework projects have contributed to the improvement of mathematics and science 
education in their states and to explain their success and limitations. The final report has 
the additional charge to use findings from the evaluation of the 16 State Curriculum 
Frameworks Projects to better understand the implementation of standards-based reform 
at the state and district levels. To accomplish this purpose, the report includes key 
findings from the First and Second Interim Reports, updates and extends those findings, 
and presents new findings from the last year of data collection activities. 

Data collection activities over the course of the 4-year study (1993-1997) were 
timed to maximize our understanding of both the development and implementation of the 
projects’ products. During the 2nd, 3rd, and 4th years of the study, data collection 
activities included: 

• Review of State Curriculum Frameworks Projects documents, including original 
proposals, continuation proposals, draft and completed framework documents, 
drafts and completed documents from model guidelines for teacher education 
and certification, criteria for recertification and model professional 
development, and available evaluation materials. 

« Review of state data from a variety of secondary sources (see complete list of 
sources in Appendix A). 

• Telephone interviews with project directors, state officials, SSI directors, 
Eisenhower state coordinators, and key participants. 

During the 2nd and 3rd years of the study, we also worked with a group of outside 
educational experts to evaluate the quality of the framework documents. In addition, we 
conducted 2-day site visits to a sample of four states (Florida, Louisiana, Nebraska, and 
Oregon) during the 3rd year of the study. In the 4th year of the study, we conducted more 
intensive week-long site visits to 8 of the 16 states. Those site visits included in-depth 
interviews with state officials as well as teachers and district officials in a sample of two 
to three districts in each state. A complete description of the evaluation methods is 
presented in Appendix B. 

In addition, the evaluation team was able to leverage data from other related studies 
conducted by SRI and its partners. We purposefully designed the data collection 
activities to take advantage of the national evaluation of NSF’s Statewide Systemic 
Initiatives (SSI), the evaluation of the American Association for the Advancement of 
Science’s Project 2061, the evaluation of the Pew Network for Standards-Based Reform, 
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and the analysis of the quality of curriculum frameworks carried out by the Council of 
Chief State School Officers. This coordination resulted in benefits and significant 
additions to the data collected through this contract’s funds. First, our coordination 
efforts allowed us access to internal case study reports on 10 of the State Curriculum 
Frameworks Projects that were located in states that also had SSIs. These case studies 
were particularly valuable because all SSI researchers had been trained to include 
questions about curriculum frameworks in their interviews. The coordination efforts also 
took advantage of overlapping staffing among the studies, allowing some members of the 
study team to visit states (via the SSI evaluation) that were not among our eight-state 
sample, as well as districts (via the Pew evaluation). Our coordination efforts with the 
CCSSO study allowed for the use of an important analysis of a sample of existing 
frameworks, as well as baseline data on frameworks in all 50 states and the District of 
Columbia. Exhibit 1 illustrates the data collection activities conducted in each of the 16 
project states. 

Exhibit 1 

DATA COLLECTiON ACTiViTiES 



State 


1996 Site 
Visit 


1997 Site 
Visit 


Document Review 
and Telephone 
Interviews (1995-97) 


Expert 

Review 


Site Visits 
through Other 
Studies* 


Alaska 






X 


X 




Arizona 






X 






Arkansas 






X 


X 


SSI 


Delaware 




X 


X 


X 


SSI, Pew 


District of 
Columbia 






X 






Florida 


X 


X 


X 


X 


SSI 


Louisiana 


X 


X 


X 


X 


SSI 


Maine 




X 


X 


X 


SSI 


Massachusetts 




X 


X 


X 


SSI 


Michigan 




X 


X 


X 


SSI 


Nebraska 


X 


X 


X 


X 


SSI 


New Jersey 






X 


X 


SSI, AAAS 


New York 






X 




SSI, Pew, 
AAAS 


Oregon 


X 


X 


X 


X 


Pew 


Rhode Island 






X 


X 


SSI 


Wisconsin 






X 







*SSI = statewide Systemic Initiative (National Science Foundation) 

Pew = Pew Network for Standards-Based Reform (Pew Charitable Trusts) 
AAAS = American Association for the Advancement of Science’s Project 2061 
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Organization of the Report 



Following this introductory chapter, we present chapters on the progress of the 
projects, the development process of the projects, and the quality of the projects’ 
frameworks. These chapters include findings from the First and Second Interim Reports, 
but update and extend those findings on the basis of our final year of data collection. 
Chapter V presents our most recent data regarding the role of the framework projects in 
state policy-making. Chapter VI examines the projects’ implementation strategies 
designed to improve teacher practice. Chapter VII focuses on the impact of the 
frameworks at the district and school levels. This chapter features extended case studies 
of districts and schools that represent a range of experience with the frameworks. Finally, 
our concluding chapter reviews the key findings and raises remaining issues. 
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II. PROGRESS OF THE PROJECTS 



Nearly all states are engaged in some form of standards-based reform. Forty-eight 
states have developed or are developing standards documents in core discipline areas. 
Forty-two states have developed or are developing new assessment systems that reflect 
the new standards. Nearly every state is revisiting its other education policies in light of 
the new standards (Gandal, 1996). The 16 states awarded State Curriculum Frameworks 
Projects are no exception: all but one project completed a curriculum framework. Even 
the one state that failed to do so — Arizona — has subsequently developed state content 
standards and is proceeding to develop performance standards. 

At the same time, the experience of the 16 State Curriculum Frameworks Projects 
illustrates that it is difficult to infuse standards systematically into school reform efforts 
in a short time. The states’ experience with the so-called “other products” — model 
guidelines for teacher education and certification, criteria for teacher recertification, and a 
model professional development program — underscores that difficulty. Among the 1 6 
project states, no state completed all the other products in a way that met both the letter 
and the spirit of the grant agreement. Those states that were best able to claim some 
progress did so as a result of activities already under way in the state, but not as a result 
of the project alone. In this chapter, we describe the progress of each of the 16 State 
Curriculum Frameworks Projects. 

Progress on Curriculum Frameworks 

The State Curriculum Frameworks Projects were designed to encourage states to 
incorporate new and emerging national standards in mathematics and science into state 
frameworks. Although there was wide variation in the format and quality of the 
documents that the project states produced, 15 of the 16 states completed curriculum 
frameworks. 

As indicated in earlier reports, states focused much of their time, energy, and 
resources on the development of state curriculum frameworks. The projects are officially 
over in all the states, including the two states, Maine and Massachusetts, that received no- 
cost extensions through September 1997. (Oregon received a no-cost extension to April 
1997.) However, many states continue to review, revise, and disseminate framework 
documents. Exhibit 2 displays the status of the frameworks as of summer 1997. 



Exhibit 2 

STATE PROGRESS ON CURRICULUM FRAMEWORKS 



state 


Status 

(Summer 1997) 


Title 


Cohort 1 






District of Columbia 


Completed 


Mathematics, Science, and Technoiogy Curricuium 
Framework 


Florida 


Completed 


Science For AH Students 


Nebraska 


Completed 


Mathematics and Science Frameworks for 
Nebraska Schoois 


New Jersey 


Completed 


New Jersey Mathematics Curricuium Framework 


New York 


Completed 


Learning Standards for Mathematics, Science, and 
Technoiogy 


Rhode Island 


Completed 


Mathematicai Power for AH Students: The Rhode 
isiand Mathematics Curricuium Framework; 
Science Literacy for AH Students: The Rhode 
isiand Science Curricuium Framework 


Cohort 2 






Alaska 


Completed 


Math/Science Aiaska Framework 


Arizona 


Not completed 




Arkansas 


Completed 


The Arkansas Science Curricuium Framework; The 
Arkansas Mathematics Curricuium Framework 


Delaware 


Completed 


State of Delaware Mathematics Curriculum 
Framework; State of Delaware Science Curriculum 
Framework 


Louisiana 


Completed 


Louisiana Science Framework; Louisiana 
Mathematics Framework 


Maine 


Completed 


Maine’s Curricuium Framework of Mathematics 
and Science 


Massachusetts 


Completed 


The Massachusetts Science and Technoiogy 
Curricuium Framework: Owning the Questions 
Through Science and Technoiogy; The 
Massachusetts Mathematics Curricuium 
Framework: Achieving Mathematicai Power 


Michigan 


Completed 


Michigan Curricuium Framework and Science 
Education Guidebook 


Oregon 


CD-ROM with 
local project 
descriptions 
completed and 
released 


A Framework for Mathematics/Science 
improvement: Case Studies from Oregon 


Wisconsin 


Completed except 
for CD-ROM 


Wisconsin Frameworks in Science and 
Mathematics 
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Two of the projects’ frameworks from the Cohort 2 states (those funded from 1993 
to 1996) were not completed or have not been released. Arizona halted work on its 
framework as the result of the 1994 election of a new state superintendent and a change in 
the state’s education policy agenda. In Wisconsin, technologically more sophisticated 
products required more time to develop. In addition, political changes in the state also 
altered the project’s plans. Wisconsin’s Frameworks in Science and Mathematics project 
(FISM) survived the shifting political sands in the state by relocating its work and its 
project director to the Midwest Mathematics and Science Eisenhower Regional 
Consortium at North Central Regional Educational Laboratory in Illinois. 

Although Oregon’s framework product was eventually released, the project became 
somewhat marginalized within the state department of education because of changes in 
personnel and priorities. During the spring of 1997, department officials determined that 
the project’s CD-ROM product was consistent with the 1995 revisions to the Oregon 
Education Act and distributed it to every school in the state. Factors affecting the 
development and completion of frameworks are discussed in greater detail in the next 
chapter. 

Progress on the Other Products 

In all 16 states, the project paid less attention and devoted fewer resources to 
drafting and implementing model professional development programs, model guidelines 
for teacher education and certification, and criteria for teacher recertification. State 
progress on devising the other products as of summer 1997 is displayed in Exhibit 3. 

Note that Rhode Island renegotiated the terms of its framework grant with ED fairly early 
in the project schedule, so that it was required to produce only framework documents in 
mathematics and science. New Jersey’s grant did not require it to develop criteria for 
teacher certification. We discuss states’ progress on each of these products below. 

Model Professional Development Programs 

The Department of Education called for grantee states to 

. . . design and pilot test a model, cost-effective inservice professional 
development program for teachers based upon world-class standards and the State 
curriculum frameworks tied to those standards. Again, the work of designing these 
programs must involve collaboration among scholars and specialists, school 
teachers, and school administrators from public or private schools. In addition, 
these programs must be pilot tested in a variety of schools throughout the State. 
[Federal Register, 57(146), July 29, 1992, p. 33604] 



Exhibit 3 



STATE PROGRESS ON OTHER PRODUCTS AND ACTIVITIES (SUMMER 1997) 



state 


Model Professional 
Development 


Model Guidelines for 
Teacher Education 
and Certification 


Criteria for Teacher 
Recertification 


Cohort 1 








District of 
Columbia 


Held workshops on 
frameworks 


No document 


No document 


Florida 


Held workshops on 
frameworks 


Separate document 
addresses teacher 
education 


None 


Nebraska 


Held workshops on 
frameworks 


Separate document 
completed 


None 


New Jersey 


Held workshops on 
frameworks 


No document: distributed 
standards to IHEs 


Not part of grant 
requirement (but state is 
considering proposal) 


New York 


Piloted model professional 
development 


Separate document not 
completed 


State task force is 
considering 


Rhode Island 


Not part of grant 
requirement (held 
workshops on frameworks) 


Not part of grant 
requirement 


Not part of grant 
requirement 


Cohort 2 








Alaska 


Held workshops on 
frameworks 


Separate state document 
in progress 


Separate state 
document in progress 


Arizona 


Suspended 


Suspended 


Suspended 


Arkansas 


Held workshops on 
frameworks 


In progress; state task 
force addressing this 


Planning 


Delaware 


Model in place 


Separate document 
completed 


In progress 


Louisiana 


SSI is using frameworks in 
professional development 


No document 


No document 


Maine 


Professional development 
is addressed in framework; 
links to SSI professional 
development 


Addressed in framework 


Addressed in framework 


Massachusetts 


SSI is using frameworks to 
guide its professional 
development 


In progress: working with 
IHEs and SSI 


New state legislation 


Michigan 


Piloted model professional 
development, held 
workshops on frameworks, 
and addressed in 
frameworks 


Separate state activities 
for science and math in 
progress 


Separate state activities 
for science and math in 
progress 


Oregon 


Local professional 
development projects 


State commission revised 
rules 


State commission 
revised rules 


Wisconsin 


Held workshops on 
frameworks 


Recommendations in 
progress 


Recommendations in 
progress 




10 



26 



Few of the State Curriculum Frameworks Projects — only Delaware, Michigan, New 
York, and Oregon — formally designed, piloted, and evaluated model professional 
development programs. New York designed alternative models and then tested them in 
two sites in the state. The results were mixed: one site abandoned the pilot; another put 
in 3 years of work with teachers holding emergency certificates. Oregon encouraged 14 
local districts to plan and pilot their own professional development projects. Again, the 
results were mixed, although each of the local projects reported benefits from its 
participation. (For more detail on these examples, see Chapter VI.) 

Other states show interesting variations on the model professional development 
requirement. Maine established framework pilot sites that included financial support for 
professional development and consultants. It also conducted 2-day conferences for 
teachers based on the frameworks (three to four conferences per year for 300 to 400 
teachers at each conference). The Maine framework document has a section on 
professional development standards, and the issue may become more visible as the state 
begins serious work on revamping teacher preparation and credentialing. Some of 
Alaska’s distance learning professional development modules are close to being models. 
A pilot was conducted with math and science curriculum revision teams that included 
twice-weekly audio and video conference links with university faculty members and 
guest speakers. There is also a self-guided course for teachers and administrators. 
Participants in either distance learning course are eligible to receive university credit. 

Even though most projects did not design and pilot model, cost-effective 
professional development programs as envisioned (and required) by the Department of 
Education, most states did conduct framework-based professional development as an 
implementation strategy. Indeed, it is widely recognized in the project states that 
frameworks and standards must be linked to meaningful and sustained professional 
development opportunities. State and local respondents recognized that framework 
documents will “sit on the shelf’ unless they are distributed in connection with training 
for teachers and other intended audiences. Framework-based professional development is 
discussed in more detail in Chapter VI. 

Model Guidelines for Teacher Education and Certification 

Eight of the framework projects claimed progress toward drafting guidelines for 
teacher education and certification, but only four — Delaware, Florida, Maine, and 
Nebraska — have actually completed separate documents or addressed teacher education 
and certification in their frameworks. In these states and others, the development of these 
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documents is often due to the work of other state reforms. The establishment of new 
teacher licensure always occurred outside the framework project itself, typically through 
the state legislature or a commission mandated by the legislature. The model guidelines, 
where drafted, simply served as recommendations to those in a position to make actual 
policy changes. 

It is reasonable to assume that the frameworks will become a resource for 
institutions of higher education. At least three states reported instances of their 
frameworks being used in teacher education programs. Almost all of Nebraska’s teacher 
training institutions are using the framework document as part of their curriculum in math 
and science education courses. The evidence is less direct in Florida, but there have been 
numerous requests to the state department of education by college faculty teaching 
preservice methods courses who want to use the document in their courses. Florida’s 
framework and its teacher preparation document were sent to the deans of all the state’s 
teacher preparation institutions. In New Jersey, all teacher preparation programs in the 
state received the framework document. 

Criteria for Teacher Recertification 

Eight states report changes in teacher recertification policies. However, the 
framework projects played a minor role in the states’ changing policy environments. One 
state, Maine, addressed teacher recertification in the framework, and a separate state task 
force has begun to examine these recommendations. In Oregon and Massachusetts, state 
recertification policies have been changed, but not as a result of the framework projects. 
However, these policy changes represent fundamental shifts in states’ requirements for 
teachers’ continued professional growth. New Jersey is in the process of gathering public 
comment on its plan to link teacher recertification to the state’s core curriculum 
standards. Separate state committees in New York, Arkansas, Alaska, and Delaware are 
still working on recertification recommendations, with input from past work of their 
respective state curriculum framework committees. 

Did the Federal Funds Matter? 

As we noted at the beginning of the chapter, 48 states have developed or are 
developing standards documents in core discipline areas. Therefore, it seems likely that 
the 16 states that received Eisenhower State Curriculum Frameworks Project grants 
would have developed curriculum frameworks or other standards documents without the 
grants. However, the grants brought resources that enriched the framework development 





process. State officials insist that the grants allowed them to produce stronger documents 
and disseminate them more widely than they could have without the grants. There is no 
doubt that federal funds allowed project states more resources and therefore more time, 
professional staffing, and opportunities for professional development than was the case in 
most non-project states. 

Although the expert reviewers did not compare the quality of frameworks from 
project and non-project states, our analysis of the reviewers’ reports on 35 framework 
documents, 13 of which were from non-project states, suggests that the range of quality 
among the project states is as great as the range of quality of the non-project states. For 
example, the reviewers had equally high praise for the draft Massachusetts Curriculum 
Framework for Mathematics (a project state) and the California Mathematics Framework 
(a non-project state). Despite similar ranges of quality among project and non-project 
states, project states, in general, produced more comprehensive documents. Unlike most 
non-project states, project states tended to produce documents that went beyond simple 
lists of content standards, and often included illustrative examples. Some project 
documents also included guidance on pedagogy, equity, assessment, and other 
components of the education system. 

Even though the states’ progress on the other products was uneven, a case can be 
made that federal funds did help those states that tried to develop model professional 
development programs, model guidelines for teacher education and certification, and 
criteria for teacher recertification. However, on the face of it, the value added by this 
investment seems to be significantly lower than that of investment in the development of 
frameworks. We will discuss why this was the case in the next chapter. 

Although there is no way of knowing what the states might have done had they not 
received grants, it is reasonable to assume that the grants added value to the states’ 
standards-based reform efforts. The grants did not guarantee a high-quality or even a 
completed document or product, but every state official involved in the projects can point 
to a positive contribution of thq grant. Just how lasting those contributions will be 
remains an open question. 

Additional policy implications of the framework projects will be discussed in detail 
in Chapter V. In the next chapter, we examine the projects’ development strategies and 
the issues they encountered. 



ill. DEVELOPING FRAMEWORKS AND OTHER PRODUCTS: 
STRATEGIES AND ISSUES 



In this chapter, we describe how the states developed the frameworks and other 
products and what issues arose during the development process. First, the chapter 
focuses on framework development by delineating the technical and political challenges 
the projects faced. Next, the chapter discusses the development of the other products and 
explains why progress was so uneven. 

Developing Frameworks 

The typical framework project went through a development process with the 
following characteristics; 

• The development process was organized through the state department of 
education.^ 

• Committees of university scholars, discipline specialists, teachers, 
administrators, and occasionally business and community representatives were 
charged with developing the curriculum framework. 

• The actual writing of the document was done by a subcommittee, which brought 
successive drafts to the full committee for review and comment. 

• Typically, the development process began by reviewing national standards 
(NCTM, Benchmarks, and the National Science Education Standards), but the 
committees viewed these documents as sources of ideas and guides, not as 
something to be strictly followed. 

The use of committees and subcommittees seemed to work well, although most states 
found that the process of curriculum framework development took more time than they 
had anticipated. 

In addition to the use of committees, most states engaged large numbers of 
educators in writing and reviewing the framework documents. In Michigan, Florida, 
Nebraska, and other states, writing and reviewing the frameworks turned out to be an 
extensive professional development exercise. Many local educators were involved with 
each of these efforts, which helped the frameworks gain acceptance by their peers. For 
example, the Nebraska project included a 16-member advisory board, an all-teacher 27- 
member writing team, a 45-member higher education committee, and 273 teacher 

^ The New Jersey project was housed at Rutgers University, but the New Jersey State Department of 
Education and the State Board of Education had a major influence on the project’s product. 
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reviewers. Florida circulated monthly drafts to an external committee of alnlbst 400 math 
and science educators and other interested parties. 

Delaware also wanted to facilitate an extensive review process. The state printed 
15,000 copies of both the mathematics and science framework drafts. With only 7,300 
teachers in the state, every teacher had the opportimity to participate in the review. 
University mathematics and science departments, the business community (including 
heavy involvement from Du Pont), mathematics and science specialists from other states, 
and 32 other groups participated in the review. 

In all states, participants most directly involved in the writing of the curriculum 
framework frequently noted that they grew professionally as a result of their involvement. 
The more time individuals devoted to developing the framework, the more they seemed 
to benefit. Brief reviews or one-shot workshops rarely resulted in reports of professional 
growth by participants. 

Beyond the impact of the direct involvement of individuals, some states found ways 
to strengthen the project, help make it central to the reform agenda, and help insulate it 
from political changes. Framework planners in Maine and Massachusetts intentionally 
embedded their development processes in other state activities. In both cases, there was a 
conceptual and organizational overlap between the framework projects, the SSIs [MESA 
(Maine Mathematics and Science Alliance) in Maine and PALMS (Partnerships 
Advancing Learning for Mathematics and Science) in Massachusetts], and the 
development of other state education policies. The involvement of members of the 
framework development team with major professional development initiatives, 
curriculum development projects, or new state assessment development helped ensure 
that the frameworks played a meaningful role in state reform efforts. 

Michigan helped strengthen its project’s role in the state by deliberately tying the 
math and science portions of the multidisciplinary Michigan Curriculum Framework (and 
the state assessment program) to preexisting state documents that specified “Essential 
Skills” in math, science, and other disciplines. The frameworks’ content standards and 
benchmarks remain influential, even though the state core curriculum mandates were 
dropped in 1995. By encompassing state standards and updating earlier goals and 
objectives documents, the Michigan Curriculum Framework has remained durable in the 
face of political shifts in the state government and on the state board of education. 
Because the thrust of these efforts was already endorsed by local educators, a 
conservative state board of education approved the content standards. 
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The Politics of Developing a Curriculum Framework 

All of the framework projects illustrated the importance of attending to the political 
and educational traditions of states and school districts. Framework development 
strategies varied among states, but all were affected by the political context, either in 
initial choices about the projects or by midcourse corrections necessary to develop 
documents that were relevant to local educators and acceptable to state leaders. 

No project turned out to be more vulnerable to political changes than Arizona’s. As 
mentioned earlier, the Arizona project was halted as a result of the election of a new 
superintendent with a strong mistrust of federally funded projects like the State 
Curriculum Frameworks Project. Within the first month of the new state superintendent’s 
term, she dismissed the director of the social studies curriculum framework project and 
returned the grant to ED. Soon after, the project leadership for the mathematics and 
science framework project was reassigned. Despite the efforts of the new project 
leadership to extend the project’s life until the “politics settled down,” it eventually 
became clear that the project’s activities could not continue. Arizona did go on to use 
state funds to develop new state standards, but without the participation of many of the 
key mathematics and science educators in the state. 

There were other cases where framework projects had difficulty coping with the 
political shifts. In Washington, DC, the framework project was overshadowed by the 
organizational and political chaos engulfing the schools. In 1996, the DC Control Board 
fired the school system’s superintendent and stripped its elected school board of its 
power. In their place, a board of trustees was appointed and a retired Army general hired 
as superintendent. The new leader moved aggressively on a number of management 
fronts, but curriculum was a low priority, given the various infrastructure problems. 

Avoiding political shifts was beyond the control of the projects. In those states that 
underwent such change, there was nothing that the project leadership could do other than 
press ahead, modify the project if necessary, and hope that the transition period passed 
quickly. Of course, the development story is not over. Some states will soon need to 
revisit their frameworks — as Florida and Louisiana are already doing. In addition, the 
districts are likely to experience some of the same political disruptions that the states 
encountered as they rethink and revise their local curricula. 

The politics of framework development also plays out in ways other than changed 
leadership and changed education policy agendas. As presented in Exhibit 4, the New 

O 

ERIC 



3 



n 

C 



17 



Jersey case offers a particularly interesting view of the politics of framework 
development. 



Exhibit 4 

THE NEW JERSEY CURRICULUM FRAMEWORK PROJECT 



The New Jersey Mathematics Coalition, centered at Rutgers University, was the 
only private group to receive a State Curriculum Frameworks Project grant. The 
Coalition focused its efforts on producing standards and then a curriculum framework 
in mathematics and collaborated with the New Jersey Department of Education 
throughout the development process. The January 1995 draft of the framework 
included four sections: The New Jersey Mathematics Standards, Planning for 
Change, Implementing the Learning Environment Standards, and Implementing the 
Process and Content Standards. This study’s expert reviewers had high praise for 
the document. They wrote: “The content reflects higher expectations of students than 
found in most frameworks and standards documents. ... It is very demanding and 
powerful as a core curriculum.” 

As the document was being developed, the governor and the commissioner 
called for a complete set of state standards to be used to define the state’s fiscal 
responsibility under its constitutional obligation to provide for “a thorough and efficient 
education.” The idea was to use standards to settle the continuing litigation over the 
school financing system. One result was that the standards (not the framework) 
needed to be approved by the State Board of Education in a high-stakes atmosphere. 

The Board made significant changes to the 1995 draft standards. Each 
approved standard is stated more briefly than the draft standards. More significantly, 
most references to the process for achieving the standard have been dropped, as in 
these examples: 

Draft: All students will develop their understanding of numerical operations 
through experiences which enable them to construct, explain, select, and 
apply various methods of computation including mental math, estimation, 
and the use of calculators, with a reduced role for paper-and-pencil 
techniques. 

Approved: All students will understand, select, and apply various methods of 
performing numerical operations. 
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Exhibit 4 (Conciuded) 



Draft: Posing and solving problems will be a major focus of all students’ 

mathematical activity so that through working with interesting, engaging, 
and intellectually stimulating situations, they come to understand 
mathematics and use it effectively. 

Approved: All students will develop the ability to pose and solve mathematical 
problems in mathematics, other disciplines, and everyday experiences. 

According to New Jersey officials, the State Board required that the standards be 
measurable. If the standards were to be used to define a “thorough and efficient” 
education, it is understandable that the Board might not want to commit, for example, 
to guaranteeing “interesting, engaging, and intellectually stimulating” learning 
environments. On the other hand, the study’s expert reviewers of the adopted 
standards were sharply critical of these and other changes. As they wrote: 

Overall, the new versions of the standards seem more sparse, utilitarian, and 
geared for easy assessment. There has been a corresponding loss in the 
powerful conception of mathematics to be communicated and in the 
emphasis on developing meaning for children and developing their 
understanding of mathematics and its power. This goes along with a 
deliberate avoidance of anything smacking of affective concern on the 
ground that this rrtight be seen as imposing values on students. 

Despite the changes in the standards, the framework still includes the rich 
vignettes and a powerful conception of mathematics. Overall, leadership of the New 
Jersey Mathematics Coalition believes that, despite the changes in the standards, the 
framework project was a success. They noted that they fought hard to preserve 
much of the language of the original draft, but that the New Jersey Department of 
Education wanted “terse language that the public can understand.” “Things could 
have been a lot worse.” 



The New Jersey example highlights two important themes relevant to the politics of 
curriculum framework development. First, the project states were careful to avoid public 
controversy, thereby creating a tension between consensus building and reform. Thus, 
the state departments (and state boards in some states) faced the challenge of promoting 
change while avoiding discord. This delicate balance required the use of language that 
the public understood in order to reduce the chances of a negative reaction, but risked 
getting no reaction, thereby weakening support for reform. 



Oregon officials found themselves confronting the tension between consensus 
building and reform when the state legislature modified the 1992 Oregon Education Act. 
Suddenly, terms like “portfolio” were dropped from the education department’s lexicon. 
In Nebraska, the state’s long tradition of local control of educational decisions meant that 
centrally developed guidance was not always well accepted in the 687 districts across the 
state. Recognizing this fact, the Nebraska project team decided early on that the only 
document that would be influential in local communities was one that was written 
primarily by teachers and not by state department officials. 

A few projects were more proactive and made significant efforts to build public 
support. The two most ambitious efforts were in New York and Louisiana. In New 
York, dissemination of frameworks and standards included the development of a video 
series for broadcast on public television and presentations by state department officials at 
public forums. Each year the project produced three videos, for a total of nine videos 
covering topics such as “Why do we need higher standards?”, “Whose schools are these 
anyway?” (parent video), “Assessment: How do we know what kids know?”, and “Road 
to reform: exemplary programs in New York State.” 

In Louisiana, the project was folded into the state’s overall standards-setting effort, 
and officials tried to build public support for standards. State officials refined and edited 
existing standards into a newspaper tabloid version that was distributed throughout the 
state for a comprehensive “public review.” The Council for Better Louisiana helped 
coordinate a public relations campaign to accompany the distribution of these reader- 
fnendly documents, including research on reactions to the standards-setting effort. 
Department staff traveled across the state, meeting with varied groups and showing a 
video on “Why Schools Need to Change,” which presented a compelling story of the 
need for school change from a successful business advocate for schools. 

However, even where educating the public was part of the project, the state 
departments avoided violating local-control sensibilities and politically contentious 
opportunity-to-leam standards, and used familiar and reassuring language. It is too soon 
to tell whether the states will be as successful in gamering professional and public 
support for high standards as they were at avoiding controversy. 

A second theme in the politics of curriculum framework development follows from 
the first: One result of avoiding controversy was that standards and framework 
development became a professional, not a public, activity for a limited number of 
professionals. Of course, defining what students should know and be able to do, and 



communicating the most effective ways for teachers to help students meet those goals, 
takes professional expertise. But gaining public and widespread professional acceptance 
requires more than creation of a high-quality curriculum framework. As we reported in 
the First Interim Report, those who participated in the development of a curriculum 
framework benefited most from the document because they had the opportunity to study, 
analyze, and reflect, giving meaning to the standards. As we discuss later, the 
implementation of standards and frameworks requires equivalent opportunities for 
teachers, administrators, and the various publics to understand the meaning and 
implications of the documents. Not everyone could participate in the development 
process; writing a curriculum framework was necessarily a limited professional activity. 
But the challenge for states and local districts was to avoid the temptation to seek 
shortcuts to implementation. 

Just as the educators who wrote the frameworks needed the opportimity to better 
understand standards, all teachers need an equivalent opportunity to make the standards 
more meaningful. Similarly, obtaining public support for standards is likely to require 
varied educational opportunities for the many publics to more fully understand standards 
and their implications. 

Developing the Other Products 

The curriculum framework states have followed very different development 
strategies in addressing the requirement that they develop model guidelines for teacher 
education and certification, criteria for teacher recertification, and model professional 
development programs. These strategies can be classified as: 

• Drafting a document or a framework chapter that discusses or recommends 
particular courses of action in these areas of policy and practice. 

• Implementing a set of activities, usually framework-based workshops. 

• Working with or handing off to an existing task force or project the 
responsibility for addressing one of these issue areas. 

• Choosing not to address particularly difficult issues. 

Unlike framework development, the states approached the development of these 
other products unevenly. What staff accepted as models of professional development 
ranged from a traditional workshop to pilot-tested programs. Certainly, the project staffs 
had uneven conceptions of their task. Beyond simple lack of understanding, however, 
there appear to be more fundamental problems. The first of these is simply emphasis and 



sequencing. The original solicitation placed more emphasis on the frameworks than on 
the other products.^ Moreover, federal and project emphasis on framework development 
is understandable, given the logic of systemic reform, which argues for policy alignment 
with the frameworks. According to that view, frameworks are developed first and then 
other policies are aligned with them. 

In addition to the sequencing of activities and the weaker emphasis placed on the 
other products, project informants repeatedly argued that they needed more time to 
complete so many tasks. Some projects did find that the process of developing a 
framework took longer than they expected, leaving little time for the development of the 
other products. More time might have helped in some states, but it is difficult to 
demonstrate that it was a factor in all cases. 

Clear Guidance and Monitoring 

Another partial explanation for the uneven progress on the other products is the lack 
of clear guidance and consistent monitoring by ED. From the start, ED funded some 
projects that lacked clearly defined activities and products associated with non- 
framework components of the grants. Throughout the life of the grants, frequent staff 
changes at ED and a lack of attention by ED leadership meant that most states had very 
little contact with ED and little or no guidance. Many states said that they had no idea 
who their project officer was. At the same time, no state complained about its 
relationship with ED. In fact, many of the informants had high praise for the supportive 
and hands-off stance of the Department. The point is that ED could have helped prevent 
such uneven progress with clearer expectations and guidance from the start, and a higher 
level of monitoring. 

Beyond the need for clearer expectations and guidance, the Arizona project suggests 
a problem with financial management and monitoring of grants at ED. The Arizona State 
Curriculum Frameworks Project got off to a slow start because of staff changes and cuts 
at the state department of education. As we will describe in Chapter V, changes in state 
leadership effectively halted work on the project. 

^ Interestingly, in response to the original announcement of the proposal for the State Curriculum 
Frameworks Projects in the Federal Register, one commentor expressed concern that the “efforts to 
develop new approaches to teacher education and certification should receive emphasis equal to that 
given to the implementation of the State curriculum frameworks” — in effect saying that the original 
guidance clearly made the other products less important. Although the Secretary responded that the 
proposed priority gave “serious and substantive attention” to the other products, the subsequent histories 
of the projects indicate that the other products did not receive the same level of effort and resources as 
framework development. 
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Throughout the fits and starts of the Arizona Curriculum Frameworks Project, 
Arizona officials requested extensions and promised a renewal of effort in a series of 
correspondences with ED. But by the late fall of 1995, it became clear that the project 
would be unable to deliver on its promises. Arizona officials claim that they received no 
response from ED to their last request to modify the project, and assumed that their 
proposal was rejected and that the funds would be automatically returned. ED records 
indicate that Arizona’s request was actually approved. 

Arizona had originally been awarded a grant in 1993 with funds made available 
through three different installments over fiscal years 1993, 1994, and 1995. Because 
Arizona had not accomplished very much by the end of the first year, ED reduced the 
amount of money that was to be made available to Arizona in the second fiscal year. 
Going into the third fiscal year, ED again reduced the amount available to Arizona 
because of the lack of progress. In the end, ED made $775,402 available to Arizona for 
its work. By the end of the grant, Arizona had not completed any portion of the project 
and had spent a total of $162,710. 

Why did ED continue to make funds available to Arizona even though Arizona had 
not made much progress? For ED, part of the problem appears to be a poor financial 
management system. Program officers in ED were expected to make continuation 
funding decisions at the end of the first and second years of the grant period even though 
they did not have easy access to the official financial documents that showed exactly how 
much Arizona had spent throughout the first and second years of the project. 

The history of the Arizona project raises questions about ED’s monitoring of the 
technical and financial side of grants. The weakness of the current financial management 
system may be solved in the near future. As part of the Education’s Central Automated 
Processing System (EDCAPS), the new Grants Administration and Payments System 
(GAPS) will replace the current tracking system in 1998. According to ED, GAPS will 
manage all grant activities under a single system approach and will provide improved 
grant information management, recipient response time, and accuracy of financial 
management information. Under the current system, payments to states are not 
reconciled with specific grant programs until the end of each quarter. GAPS will require 
that states specify which program the funds are for when they make their requests for 
payments from ED. Regardless of the effectiveness of the new financial management 
system, ED still faces the challenge of improving its monitoring system. Without 
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accurate and timely information on the progress of federally sponsored projects, ED staff 
will be unable to assist a troubled project or halt a failed one. 

The Limits of Poiicy influence by the Other Products 

One final reason that the projects made such uneven progress on the development of 
the other products has to do with the nature of the products themselves and the location of 
the projects in the education system. Frameworks can serve as policy guidance, but 
teacher education, certification, recertification, and (to a lesser extent) professional 
development are driven by specific policies. Thus, the areas addressed by the other 
products are policy sensitive and of concern to a set of interested parties (IHEs, teacher 
unions, etc.). If the projects were to produce guidelines, criteria, and models that had 
credibility and were likely to influence policies in these areas, they could not simply 
produce theoretical documents in isolation from the interested parties and the politics 
associated with each of these potentially controversial policy areas. The projects were 
faced with a difficult task, compounded by the fact that they generally were removed 
from or unfamiliar to the institutions and individuals most invested in the particular 
policy area. 

Several examples from the framework states illustrate these themes. Like many 
other states, Nebraska was not systematically addressing the development of other 
products. Moreover, to ensure full ownership by K-12 teachers, the project intentionally 
excluded representatives from higher education until the project was well under way. 
However, when framework staff learned about a math professor at a small college who 
was interested in K-12 math education and the preparation of teachers, the project teamed 
him up with a science educator from the University of Nebraska-Omaha to write the 
teacher preparation product. The resulting document was the state’s Guidelines for 
Teacher Preparation: Mathematics and Science. 

In Alaska, the development of state standards for teachers reflects recommendations 
from the framework project. However, new certification and recertification requirements 
aligned with state standards for teachers are not complete. Oregon also revamped teacher 
licensure rules, but the effort was not related to the framework project. 

Several states had explicit or implicit rationales for ignoring or minimizing the 
development of professional development models and guidelines for teacher certification 
and teacher recertification. Rhode Island formally negotiated itself out of the requirement 
to develop other products and returned a portion of the state’s framework grant. New 
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Jersey did not include a plan to develop criteria for teacher recertification in its grant, 
pointing to the state’s lack of recertification policies. Finally, the framework committees 
in Florida originally plarmed to develop a professional development model, but 
abandoned the plan after determining that local needs varied too widely to be addressed 
by a single model. Florida uses a variety of workshop modules to disseminate the 
framework and its most visible “other product”: the Curriculum Plarming Tool. 

Conclusion 

Thus, the development or lack of development of the other products was a 
disappointing component of most of the projects. Most states focused on framework 
development, neglected development of the other products, or ceded responsibility for the 
other products to official task forces or commissions. However, the root cause of the 
projects’ uneven progress on the other products rests with unrealistic assumptions of the 
original solicitation. Given rather small sums of money (averaging $850,000 per project) 
and a limited time (3 years), the four-fold agenda was overly ambitious. In the future, ED 
might consider avoiding such multifaceted and complex awards. Alternatively, if ED 
targeted specific activities in multiple awards and required that the projects coordinate 
their efforts, or gave clear direction as to the level of effort expected on each facet of the 
grant, the states would be better able to complete all components of the project. 

In Chapter IV, we examine the quality of the curriculum frameworks that were 
developed by the projects. 
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IV. QUALITY: A REVIEW OF MATHEMATICS AND SCIENCE 
CURRICULUM FRAMEWORKS 



The State Curriculum Frameworks Projects have developed a set of curriculum 
frameworks in mathematics and science. But how good are these documents? And how 
are these state documents translated at the local level? Decisions about quality are 
difficult to make, call for specification of criteria, and ultimately rely on expert 
judgments. Consequently, from the beginning of the evaluation, the study has been 
working with a group of 18 distinguished mathematics and science educators."* This 
group of reviewers has developed a process for assessing the quality of framework 
documents and carried out such an assessment on a sample of 12 framework documents 
(see Appendix B for details on the methodology of the review).^ Following the experts’ 
analysis, our study collected 10 local curriculum documents from 5 states and examined 
their alignment with state frameworks. The results of both of these analyses are 
presented in this chapter of the report. 

The reviewers agreed that their analysis of the curriculum frameworks should be 
useful to the states. As a result, a confidential report on each state’s framework was 
written and forwarded to the respective state. For purposes of this report, the reviewers 
and the study team agreed that a cross-framework analysis should avoid criticizing 
individual state efforts, unless doing so was central to the analysis. This decision was 
made to discourage simplistic ranking of the states’ documents. 

The expert reviewers used three broad criteria to judge the quality of the 
frameworks. The first of these was consistency with the national standards: the National 
Council of Teachers of Mathematics’ (NCTM) Curriculum and Evaluation Standards for 
School Mathematics, the National Science Education Standards (NSES), and the 
American Association for the Advancement of Science’s (AAAS) Benchmarks for 
Science Literacy. The reviewers did not approach these national standards documents 
uncritically, but they did consider them useful tools for analyzing a framework’s subject 
content and pedagogy. The reviewers did not suggest that the states should simply copy 

'* This group of experts also served two related studies, an NSF-funded study of curriculum frameworks 
conducted by the Council of Chief State School Officers (CCSSO) and a study of selected textbooks for 
the American Association for the Advancement of Science’s Project 2061. 

’ The 12 documents came from 9 of the 16 curriculum framework states (Alaska, Arkansas, Delaware, 
Louisiana, Maine, Michigan, New Jersey, Oregon, and Rhode Island) and represented roughly equal 
numbers of mathematics and science frameworks. 
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the national standards, but they were concerned that key components of the national 
standards be reflected in the frameworks. 

The second criterion was usability. The reviewers examined each framework with 
an eye toward whether or not the document spoke to its audience in ways that were likely 
to be effective. The reviewers examined each framework’s language and style, its 
internal coherence, and the clarity of its message. 

The third criterion was equity. The reviewers examined the frameworks to see how 
equity issues were dealt with, assuming that clear and comprehensive guidance on how to 
facilitate high achievement in mathematics and science by all students was a necessary 
component of a good framework. 

In general, the reviewers found that state frameworks showed marked progress in 
expanding beyond a basic-skills emphasis to focus more on higher-order skills for all 
students in mathematics and science. At the same time, the reviewers found that some of 
the frameworks omitted some of the major categories of the national standards, suffered 
from a lack of usability, or failed to convey adequately how equity can be achieved. The 
reviewers’ findings are described in more detail below. 

Curriculum Frameworks and Fidelity to the National Standards 

Frameworks and National Content Standards 

All of the 12 frameworks reviewed acknowledged the influence of the national 
standards. The fact that each of the 12 frameworks had accepted the national standards is 
an important, if not surprising, finding. These diverse states’ embrace of the national 
standards lends political support to the standards-based reform effort. 

The reviewers found that many of the frameworks captured “the spirit and essence 
of the national standards and the Benchmarks remarkably well.” Some frameworks were 
more explicit than others in their efforts to demonstrate alignment with the national 
standards. For example, one group of reviewers noted that Maine’s framework provided 
a chart that clearly places the document in the context of the national standards 
documents (see “Connecting Maine ’s Curriculum Framework for Mathematics and 
Science with Maine ’s Common Core of Learning and National Standards” in 
Appendix C). Rhode Island was also specific. The Science Framework for Rhode Island 
delineated the principles that it adapted from the NSES and indicated that “the science 
benchmarks in this document are adopted or adapted from Project 2061.” Other states 
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simply stated that their frameworks reflected national standards. For example, Louisiana 
stated that “the Louisiana Mathematics Curriculum Framework reflects national 
standards in defining 
K-12 curricula” (p. 6). 

The reviewers found that exemplary frameworks were those that not only reflected 
the national standards, but went beyond them. The draft New Jersey Mathematics 
Curriculum Framework is a good example of a document that goes beyond the NCTM 
standards. The reviewers cited New Jersey standards #1 and #18: 

1 . All students will be expected to demonstrate high levels of mathematical thought 
and will have opportunities for ftirther study in mathematics at all grade levels, 
including courses beyond traditional geometry and algebra in high school, so that 
they are continuously challenged to maximize their achievement. 

18. All students will develop their understanding of the conceptual underpiimings of 
calculus through experiences which enable them to describe and analyze how 
various qualities change, to build informal concepts of infinity and limits, and to 
use these concepts to model, describe, and analyze natural phenomena. 

(Chapter 1, pp. 16-17) 

As the reviewers noted: 

The content reflects higher expectations of students than found in most framework 
and standards documents, especially in secondary schools. Where NCTM hedges 
on mathematics for all students 9-12, this framework extends much of the 
mathematics for only college-bound to all students, especially the conceptual 
underpiimings for calculus. 

... It is very demanding and powerful as a core curriculum. 

As we discussed in the preceding chapter, a subsequent review of the New Jersey 
document after the mathematics standards were revised and approved by the New Jersey 
Board of Education found that the revised standards lost some of their power to 
communicate the vision embodied in the earlier draft. Despite the revisions to the 
standards, the remainder of the framework document preserved the characteristics that the 
reviewers found valuable. Given the reviewers’ concerns, much of the success of the 
New Jersey mathematics reform effort will depend on teachers’ use of the entire 
framework document, not just the state-approved standards. 

The reviewers found that 5 of the 12 frameworks fell somewhat short of the national 
standards. Three of the six science curriculum frameworks and two of the six 
mathematics frameworks omitted or collapsed major content categories that appear in the 
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national standards documents. In science, those omissions typically included science and 
technology, science in personal and social perspectives, history and nature of science, 
health, and mathematics. In mathematics, two of the frameworks omitted or collapsed 
one or more of the following NCTM content standards: number sense, patterns and 
functions, communication and reasoning, discrete mathematics, and the underpinnings of 
calculus. 

The reviewers also found examples of frameworks that used the major categories of 
the standards, but the text designed to explain the fundamental concepts and principles 
that underlie the standard has been rewritten in such general language that much of the 
meaning is lost. In one instance, the reviewers compared a content standard from the 
NSES with one from a science framework. From “Guide to the Content Standard” in 
NSES (p. 127): 



Light, Heat, Electricity, and Magnetism 

• Light travels in a straight line until it strikes an object. Light can be reflected by a 
mirror, refracted by a lens, or absorbed by the object. 

• Heat can be produced in many ways, such as burning, rubbing, or mixing one 
substance with another. Heat can move from one object to another by conduction. 

• Electricity in circuits can produce light, heat, sound, and magnetic effects. 

Electrical circuits require a complete loop through which an electrical current can 
pass. 

• Magnets attract and repel each other and certain kinds of other materials. 



By contrast, the state science framework stated: “Identifying and describing the 
differences in the production and properties of light, heat, sound, electricity and 
magnetism.” The reviewers argued that in this example the national standards had 
become diluted by generalization, and they doubted that “a teacher or other practitioner 
could make use of them and certainly could not develop assessment tasks to measure the 
stated standard.” 




Frameworks and Other National Standards 

The reviewers noted that all but one framework went beyond content standards and 
directly or indirectly addressed other standards, such as those for teacher preparation and 
professional development, as well as program and system standards. In those cases 
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where the frameworks were most explicit about the other standards, the reviewers were 
often laudatory, as in their review of the draft New Jersey Mathematics Curriculum 
Framework: 

. . . the framework provides examples and models for achieving the four learning 
environment standards of equity, assessment, technology, and student behavior and 
attitudes. ... it includes a sophisticated analysis of the requirement for realizing 
systemic reform. As the document points out, it is not enough to think about 
professional development; strategies must also address the broad spectrum of 
policies and issues. While national documents and most other frameworks do not 
take seriously the school as an organizational imit, this framework does. It also 
includes a discussion of the costs involved in incorporating technology. 

In examining the 12 frameworks as a whole, the reviewers foimd that the states paid 
more attention to content standards than to other standards, such as teaching standards. 
However, the teaching standards were sometimes reflected in the vignettes, as in this 
example from the Delaware Mathematics Curriculum Framework (p. 45): 
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Other frameworks offered a valuable set of curriculum ideas and activities. How- 
ever, many fell short of offering the kind of rich examples of good pedagogy necessary to 
concretely communicate the teaching standards found in the NCTM standards or in the 
NSES. In Exhibit 6, the reader is offered a compelling view of classroom learning, but 
the teacher is absent from the scene. However appealing this portrait of student learning 
is, it offers educators few clues as to what the teacher is supposed to do to facilitate such 
learning. 



Exhibit 6 

IN THE CLASSROOM 



Fourth-graders are exploring estuarine soils and plants. After a field trip to the 
estuary to collect soil samples, the students examine the various soils, noting 
characteristics of each one (texture and amount of organic matter). 

The class discusses student observations and matches soil descriptions (peat, 
mud, or sand) with the soils. To learn which characteristics enable plants to succeed 
in particular soil types, groups of children check the soils for water percolation rates. 

Next, students synthesize what they have learned through investigation, 
research, and discussion by “designing” a plant that could live in each of the soil 
types. Finally, the groups present their plant designs to the rest of the class. The 
plant may be fictitious or real, but students must explain how the plant is able to get 
the appropriate amounts of nutrients, gases, and moisture from the soil. The group 
agrees to investigate further by growing some real plants in each of the different soil 
types. 



Frameworks and Assessments 

The reviewers found that all the frameworks attempted to address issues associated with 
assessments in some fashion, usually classroom-level assessments. Typically, the states 
described the different kinds of assessment, defined the purposes of assessment, and 
established qualities of a good assessment system. As the Science Framework for Rhode 
Island explained: 

Many different methods of assessment should be used to assure that all students — 
those with various abilities, backgrounds, and levels of English language 
proficiency — have ample opportunity to be challenged by assessment. Moreover, 
assessment should be an integral part of the learning process, not the end result. An 
assessment program for students should: 

• be coherent and comprehensive; 
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• be equitable and engage all students; 

• be integrated with instructional strategies and curriculum materials to 
promote effective student learning; and 

• provide information that will help yield valid inferences about students’ 
learning. 

A few states offered concrete examples of performance indicators in their 
frameworks. For example, under the Spatial Sense and Geometry Standard in Delaware’s 
mathematics framework, the reviewers pointed to the following performance indicators: 

Through the investigation of meaningful problems, individually or in cooperative 
groups while using appropriate technology, all students in grades 6-8, building upon 
the K-5 expectations, will be able to: 

8.60 identify, describe, compare and classify two and three dimensional 
figures; 

8.61 use a compass and straight edge as tools for basic geometric 
constructions; 

8.62 investigate and discover geometric relationships through the use of 
manipulatives, constructions and computer graphic software; 

8.63 create models of sets of three dimensional figures such as a cube, 
rectangular prism, cylinder and square pyramid; 

8.64 visualize and draw orthographic projections; 

8.65 discover and apply geometric properties and relationships such as 
congruence, similarity, parallelism, perpendicularity and symmetry; 

8.66 apply geometric properties and relationships to make conjectures. 

However, the reviewers noted that some of the performance indicators seem “more 
like behavioral specification of the performance related to general standards and do not 
have criteria that would help evaluate performance quality.” They also noted the 
introduction of scoring rubrics in the second volume of the framework, suggesting “the 
kinds of performance benchmarks that Delaware seems to be seeking to establish.” 

The reviewers agreed that developing good and appropriate performance 
assessments is a difficult task, one that requires substantial professional development for 
teachers. At the same time, some reviewers questioned the fundamental wisdom of 
relying on performance assessment as a prominent, large-scale measurement strategy. On 
the other hand, the reviewers pointed out, embedding performance items into classroom 
instruction is a sensible step. 



49 

o 

ERIC 



34 



Usability of the Framework 



The second criterion the reviewers applied was the usability of the framework. 
Regardless of how well aligned the document might be with national standards, its impact 
on policy-makers and practitioners requires that they can read and understand it and use it 
to alter their practice. The reviewers examined each framework’s style and presentation, 
its internal coherence, and its use of language. 

Style and Presentation 

Some frameworks were farther along in the production process than others, but the 
reviewers clearly appreciated those documents that paid attention to style and 
presentation. After analyzing Maine ’s Curriculum Framework for Mathematics and 
Science, the reviewers reported: 

The format, coherence, and readability should serve as a model for other states. The 
message is clear and consistent. This is not intended to be a how-to manual, rather 
it presents a vision that is comprehensible and coherent — a standard to follow. This 
framework provides extensive concrete examples, vignettes, and snapshots which 
develop the vision thoroughly and help people think about how they might apply 
this vision in a variety of settings. 

The reviewers also found that some of the frameworks were of such length and 
format that they were overwhelming to the reader. Indexes and graphic organizers were 
often missing from the frameworks. Some frameworks were attempting to use 
technology to help ease the use of long and complex guidance. Oregon’s framework was 
to be produced on CD-ROM in an attempt to make it a richer “living document.” The 
reviewers saw this approach as innovative and promising, but asked how Oregon would 
ensure teachers’ access to computers to view the framework. 

Internal Coherence 

Maintaining internal coherence in a dociunent that is often produced by committee 
is a major challenge facing all framework developers. The strongest frameworks excelled 
in this area. The reviewers described the draft New Jersey Mathematics Curriculum 
Framework as such: 

The examples in the framework are consistent with the principles. The vignettes in 
chapter one were crafted to illustrate multiple standards in realistic classroom 
settings, rather than presenting a single vignette for each point and standard. These 
vignettes explicitly demonstrate the standards being addressed. The result is some 
very compelling pictures of what classroom practice and student engagement might 
look like if the framework were implemented. 
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A sample vignette from the New Jersey Mathematics Curriculum Framework is presented 
in Appendix C. 



ERIC 



The reviewers noted, particularly among the science frameworks, that some states 
did an exemplary job of reflecting the national standards and illustrating them through 
local examples. For example, the reviewers found that Delaware’s science framework 
offered extensive recommended activities that involved local conditions and issues. As 
they reported; “Some of the content topics, for example, polymer chemistry and the 
Delaware estuary, reflect local industry and enviroiunent, and we find that to be very 
appropriate in the context of a small state.” 

Some frameworks made significant efforts to point teachers to further assistance. 
The Science Framework for Rhode Island not only included a bibliography at the end of 
each section, but it also offered a chapter on resources for teachers, including lists of 
local, regional, and national organizations and selected curriculum projects. In addition, 
the Rhode Island framework directed teachers to What ’s Out There? A Directory of 
Science and Mathematics Resources for Educators. 



The reviewers also acknowledged just how difficult it is to develop examples and 
vignettes that effectively communicate the standards. Reviewers had both high praise 
and suggestions for the examples in Maine’s framework. Although they found the 
“Snapshot” and “Measuring Up” sections that accompanied the main text to be “distinct 
strengths of the document,” they did not always find the examples well aligned with the 
text. They found the vignettes provided in the “Beliefs about Best Practices” a “style of 
examples [that] should be emulated by other frameworks,” but wondered why more 
vignettes were not found in the other sections of the document. An example of one of 
these vignettes is presented in Appendix C. 



The reviewers made similar comments concerning the vignettes in the draft 
Mathematics/Science Alaska Curriculum Framework. As they reported: “Within the 
context of well-done vignettes, we observed that they could have improved if, 
throughout, the relevant situations used in the vignettes were more clearly annotated as to 
the specific discipline they are connecting.” The reviewers found that the vignette in 
Exhibit 7 “would benefit from giving the reader better guidance about how to end the 
directions.” In addition, the reviewers asked: “. . . how does the teacher make decisions 
about questions and guiding instruction? In bold, the annotations should have stated the 
‘big ideas,’ clarifying the specific mathematics the teacher is getting at in the questioning. 
What was the teacher’s thinking? Why did she choose these particular topics?” 
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Exhibit 7 

MATH - iNTERMEDiATE LEVEL: DOG MUSHiNG 



Math Standards: Connections; Content 

The teacher is using open-ended questioning strategies, making connections to 
real-life and to local culture, and using a graphic organizer (KWL Chart). 

It is 8:00 am. Mrs. Titus greets her students at the door. The students are 
studying dog mushing. Today they will be discussing how to find out how much it 
costs to raise a team of dogs. 

Mrs. Titus: How many of you have a dog? (Show of hands.) What do you know 
about how much it costs for your family to keep a dog? 

Student: I think we buy 50 lb. of dog food every two weeks. 

Mrs. Titus: Are there other costs in caring for a dog? 

Student: Every once in a while we have to take the dog to the vet. 

Mrs. Titus: Do you know how much it costs to take your dog to the vet? 

Student: No. 

Mrs. Titus: Would it be possible for you to find out? If you were raising your dog 
to race in the XYZ dog race, do you think there would be other costs? 

Student: I know a musher; I could ask. 

The teacher makes a KWL chart during this discussion and fills out the “What 
We Know” section. She then solicits questions from the students for the “What We 
Want to Learn” section. 

The student invited the musher to come into the classroom to answer questions 
about raising dogs. 



In another example, the reviewers found the sample activities that accompanied 
each standard in the Delaware Science Curriculum Framework'. 

... to be very process skill oriented and good. They are substantial and specific. 
For example, on page 29, accompanying substandard 1 on light is a sample activity 
involving experimenting with different materials to determine how different 
materials react to light. Another example, on page 70, accompanies a substandard 
on genetic engineering with an activity involving a classroom debate on bioethical 
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issues. It is also good that the document indicates that these are sample activities, 
so teachers will feel free to modify them. 

However, the reviewers also worried that “many of the activities could be interpreted in a 
cookbook fashion.” The reviewers cited the sample activity on page 27 that suggests 
students “investigate the influence of the sun on temperature. For instance, record and 
compare air and water temperatures at day and night, temperatures at various times of the 
day, and temperatures on a cloudy day.” The reviewers feared that a teacher might 
simply give students worksheets and ask them to collect data, arguing that the 
“framework does not work the example in a constructivist way. For example, suggesting 
that the teachers encourage students to come up with and test their own hypothesis about 
how the sun affects temperature.” 

The reviewers also found that, in terms of both style and content, many of the 
frameworks suffered from a lack of internal coherence. These problems were usually 
found in the examples or vignettes. One group of reviewers criticized one framework for 
its examples: 

The examples lack commentary that could help the reader identify the specific 
content objectives and specific student or teacher actions to receive special notice. 
The role of the mathematics content in some examples is not clear. For example, a 
middle school computation and estimation unit constructing a model city, that 
requires 5 to 7 weeks, does hot specify what computation or estimation objectives 
are addressed. The teacher is fairly invisible in many examples, so the opportunity 
to illustrate the vision of good teaching has been lost. 

The reviewers also found that the absence of concrete examples may send the 
wrong message to teachers. In one case, the reviewers found that: “The examples do not 
consistently illustrate principles of the standards (i.e., reasoning, problem solving, 
communication), leaving the impression that they are not consistently supported.” 

The Language of Reform 

The reviewers argued that the frameworks, along with much of the current 
standards-based reform effort, suffered from the use of rhetoric that lacked precision and 
was susceptible to misinterpretation and sharp criticism. Although educators are often 
accused of using buzzwords and speaking in vague terms, the reviewers went farther to 
argue that a number of central terms are confused in the national reform rhetoric in 
mathematics and science education or are used naively without consistency across 
frameworks and documents. These terms include vision, goals, objectives, curriculum 
standards, content standards, and performance standards. The reviewers argued that 
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these terms and others are used in an inconsistent, overlapping way both among and 
within frameworks. They called for clarification and guidance for consistent use of these 
terms. 

Some frameworks received specific criticism regarding the language they used. In 
one case, the reviewers reported: 

One issue that needs to be resolved is the use of the terms standards, benchmarks, 
goals, and objectives. The original document set forth a set of goals and objectives, 
which appear to be the same as the draft standards. The standards appear to be what 
is traditionally considered as content standards. A key is needed somewhere to help 
the user to determine how these terms are related to each other. Consistency of 
language across multiple documents from the same source is important if the 
message is to be understood. 

More generally, the reviewers objected to the misuse and overuse of the term 
equity. The reviewers asserted that equity involves a variety of concerns that are often 
lumped together and used without differentiation. They pointed out that the frameworks 
and the reform movement in general use the term equity carelessly to refer to ethnic 
minority and underrepresented groups, gender issues, equality of access, equality of 
outcomes, equality of resources for various subpopulations, the multicultural basis for the 
development of science and mathematics, and/or attention to groups with different 
interests and learning styles (including individual learning versus learning in groups, and 
so forth). 

In addition, the reviewers argued that a number of terms from pedagogical 
concerns, models of learning, and styles of teaching are also used in unclarified forms. 

For example, such terms as constructivism, conceptual understanding, inquiry-based 
learning, activity-based learning, relevant, applied, realistic, and real world have come 
to cloud rather than clarify what is meant. The reviewers maintained that many of these 
terms are not bad in and of themselves, but they are often used ambiguously, 
inconsistently, or without precise meaning and thus detract from discussion rather than 
furthering it. In particular, constructivism is often interpreted to mean that young 
children construct their own science (or mathematical) knowledge de novo and is a 
misreading, if not a destructive interpretation, of research on the epistemology of science 
and the construction of canonical knowledge by scientists and mathematicians. 

The reviewers went on to argue that careless use of language and the use of cliches 
often raises concerns, red flags, and sniping from opponents of reform, and even from 
supporters. Citing the recent attacks on NCTM standards, the reviewers noted that 
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cliches are often used as shorthands for positions and platforms and are, therefore, easy 
targets for critics. They pointed out that such cliches may also lead to uninformed 
support for reform efforts and may be used as a substitute for clear thinking. By using 
terms currently in vogue, educators may give the impression that an individual document 
is the product of careful thinking, when in fact the use of cliches becomes a substitute for 
understanding. 

Finally, the reviewers also argued that some reform rhetoric reflects a mechanistic 
model of designed change and implementation. The language often describes reforms as 
coming on-line or being rolled out, ratcheted up, or scaled up. The current language of 
reform has an engineering vocabulary, apparently replacing educators’ earlier reliance on 
medical terms like diagnose and remediate. The reviewers found that the fi’amework 
writers speak of leveraging, bridging, linking, forging, and operationalizing. Reformers 
build fi'ameworks, disseminate them, and implement them. Such language suggests 
linear systems with causally linked outcomes in the form of improved student 
performance. The reviewers argued that, in fact, reform is a complex social and political 
process for which more organic, less mechanistic models and language are appropriate. 
Reform implies growth and change, and is more like biological change — ^reflected in 
diverse ways. 

Equity 

The third major criterion that the reviewers used in their analysis was that clear and 
comprehensive guidance on how to facilitate high achievement in mathematics and 
science by all students was a necessary component of a good fi’amework. The reviewers 
found that every fi’amework under review made mention of equity issues. However, the 
reviewers were critical of this component in all the fi’ameworks, including those with the 
strongest equity components. After analyzing the New Jersey Mathematics Curriculum 
Framework, one group wrote: 

We would like to reinforce the point that the treatment of equity in this fi’amework 
far exceeds anything we have seen anywhere else. The fi’amework includes a 
chapter on equity, and makes fi-equent references to all students and equity issues 
throughout. . . . However, there is room for improvement. . . . Although the equity 
chapter is superior to anything else written on equity, to the extent that vignettes 
carry part of the message, they fail to convey how equity can be achieved. 

Other groups of reviewers analyzing other fi’ameworks echoed the finding that the 
fi’ameworks lack concrete examples and vignettes designed to deal with “the cognitive 



implementation of equity-sensitive teaching in the classroom.” One group of reviewers 
foimd that the draft of the Alaska Mathematics and Science Curriculum Framework stood 
out for the thoroughness and forthrightness with which it addressed the equity issue. 

They cited the introduction to the equity section of the framework: 

Schools can no longer ignore the interaction of culture and school success. 

American schools have unjustly favored the students of the dominant culture by 
ignoring the norms and expectations of other cultures. Culture can be defined as the 
learning beliefs, imderstandings, world view, and norms acquired through ethnic, 
racial, lifestyle, gender, physical disability, or other group identities. Schools must 
provide equitable opportunities for students of all culhnes. . . . Educational equity 
means more than the provision of equal access to courses, facilities, and programs. 

It also means designing schools to meet the diverse educational needs of all students 
through equitable school finance, the school commimities, family empowerment, 
teacher preparation and training, and assessment. {Alaska Mathematics and Science 
Curriculum Framework, Draft, January 1996, pp. 15-16) 

Although the reviewers were pleased to see such language, they argued for more. 
“Combining the sociology of equity issues with more detailed knowledge about 
children’s cognition — going beyond common learning style vernacular — ^would take an 
already strong equity statement one important step further.” 

Although generally critical of the frameworks’ treatment of equity issues, the 
reviewers acknowledged the lack of models of classroom vignettes that give advice and 
guidance on how to seriously address equity concerns. Referring to one framework, the 
group reported: “There are only two pages on equity. This is still more than is done in 
the NCTM standards, but this is no recommendation.” 

Local Curriculum Documents 

The reviewers predicted that local contexts would mediate how the frameworks 
were actually used, regardless of the quality of the documents. Our subsequent site visits 
to local districts supported this prediction, as we discuss later, but we also discovered that 
local translation of state documents varied widely. 

In the final year of the evaluation, we gathered 10 local math and science 
curriculum documents from school districts we visited in 5 project states. In our analysis 
of these documents, we foimd great variation in the quality of local documents and in 
their adherence to the state curriculum frameworks. Some local documents described 
themselves as curriculum frameworks; others were called curriculum guides. The vast 
majority acknowledged their state’s curriculum frameworks and described their local 



document as derived from the state’s documents. The primary purpose of the local 
documents was to present local standards and sample curriculum units. An example of 
science standards and sample units for the 4th grade is presented in Appendix C. 

Local documents varied considerably in their level of detail, especially with regard 
to sample activities associated with standards or units. Some local curriculum 
frameworks and guides had no sample activities; others offered extensive sample lessons 
linked to standards. Similarly, the local documents varied in the amount of guidance they 
offered in other areas, such as assessment and instructional techniques. 

Most local documents we examined contained content standards that corresponded 
to the content standards in the state framework. Although there was some variation in 
how the state standards were presented, it was possible to find the state standards 
reflected in the majority of local documents we examined. But, as was the case with the 
national standards and the state frameworks, some local documents were more explicitly 
aligned with the state frameworks than others. In a few cases, the local documents did 
not seem to reflect the state frameworks at all. 

High-quality state curriculum frameworks appear to help districts produce high- 
quality curriculum frameworks or guides; however, they do not guarantee quality. For 
example, Massachusetts’ highly regarded curriculum frameworks were the basis for the 
development of two local curriculum documents that we examined. In one case, the local 
document carefully followed the state’s mathematics and science and technology 
frameworks, and presented detailed lessons that helped illustrate the intention of each of 
the state frameworks’ strands. In the other case, the main strands were presented, but 
they were followed by extensive lists of objectives that seemed to reflect the district’s 
traditional scope and sequence rather than the state frameworks’ emphasis on depth over 
breadth. In this case, Cohen and Spillane’s (1994) prediction that “reformers will do 
better at addition than subtraction” and that they will be less able to “reduce the clutter of 
previous programs and policies, or to fundamentally change teaching” appears to have 
been realized. 

Conclusions from the Quality Review 

The discussion about the implications of these findings led the reviewers to a 
broader reexamination of the basic assumptions behind the process of analyzing 
frameworks and the place of frameworks in the education reform landscape. 
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First, the reviewers’ discussions about cross-framework quality were tempered by 
the realization that a state’s framework may or may not represent great movement on that 
state’s part. Even frameworks that the group believed had many worthwhile features 
need to be analyzed in terms of their impact. For instance, despite the group’s consensus 
that the New Jersey Mathematics Curriculum Framework had many commendable 
qualities, the framework’s quality is irrelevant if it does not become part of the state’s 
reform agenda, is not widely distributed, does not inform and fit closely with other 
reform activities, and is not accompanied with the supports necessary for it to be useful to 
its audiences. Moreover, the governor’s attempts to use standards to define a “thorough 
and efficient education” in New Jersey reminded the reviewers that curriculum 
frameworks exist in a political environment. 

The reviewers’ findings regarding the quality of the frameworks have some 
important implications for the implementation of the documents. All the documents 
reflected the national standards, and as such fulfilled the minimum conditions necessary 
to bridge those standards to the state and local levels. However, the reviewers found that 
the documents varied in their usability. If the reviewers are correct about the likely 
usefulness of the various documents, one would expect to see similar variability in how 
effectively the documents are able to influence policy and practice. Their concerns about 
the language of reform suggest that broad agreement about what a framework means may 
be superficial and may make it more difficult to translate the framework’s ideas into good 
policy and practice. 

The challenge in examining the implementation of the frameworks is to understand 
their fit in the state and local contexts. Regardless of how usable the documents are, state 
and local contexts will mediate their actual use. Thus, many variables determine the 
frameworks’ influence on the different components of the education system. 

Reflections on the Quality Review Process 

The extensive state and local efforts to produce curriculum frameworks and other 
curricular guidance that reflect high standards have resulted in a great deal of interest in 
assessing the quality of those documents. The American Federation of Teachers’ annual 
reports on state standards and the various reports of the Council of Chief State School 
Officers are examples of recent attempts to assess quality . As a key component of the 
evaluation of the State Curriculum Frameworks Projects, our analysis of quality of a 
sample of framework documents has revealed some important lessons regarding both the 
process and outcomes of the quality review. 




43 



58 



Lessons on the Process 



A complete description of the methods used to review curriculum frameworks 
produced through the projects is presented in Appendix B. Each aspect of those methods 
turned out to be an important part of the overall success of the review process. However, 
three features stood out. 

First, we purposely selected reviewers who would bring multiple perspectives to the 
review process. While content specialists were invaluable in helping to determine 
whether the frameworks reflected the national standards, the mix of state officials, district 
officials, federal officials, research staff, teachers, and academics enriched the review 
process. Because of the multiple perspectives, the review process involved an analysis of 
the frameworks through many lenses, including mathematics and science content, equity, 
policy, teacher usability, etc. 

Second, we devoted extensive time to establishing criteria forjudging quality. 

Those criteria eventually came to reflect the multiple perspectives of the reviewers. More 
importantly, the criteria were specific enough to provide a common procedural approach 
to the analysis, but qualitative enough to capture the complexity of the frameworks to be 
analyzed. 

Third, the reviewers’ analysis was grounded in background information on each 
framework’s development and the state context that helped shape the document. 

Research staff provided the reviewers with the background information, and helped 
underscore the reviewers’ original premise that framework documents cannot be analyzed 
in isolation from their context. 

Of course, the review process could be improved. Although the main purpose of 
the review was to conduct a cross-framework analysis of quality, better interrater 
reliability on individual documents would have strengthened the process. Small groups 
of five or six reviewers, selected for their multiple perspectives, analyzed individual 
documents. Given more time and resources, a quality review by at least two groups of 
reviewers would have been preferable. 

Lessons on the Outcomes 

The primary purpose of the review was to assess the overall quality of the curricu- 
lum frameworks that resulted from the projects. A secondary purpose was to communi- 
cate the findings of individual reviews back to the states. In the end, those purposes were 
met. However, the quality review also revealed some unexpected outcomes. 
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Despite the reviewers’ desire to assist the individual states in improving their 
curriculum frameworks, the reviewers’ comments were not always welcome in light of 
state education politics. Particularly when the reviewers were sharply critical of a state’s 
document, those criticisms were not widely distributed by the framework project director. 
In a few cases, the reviewers’ comments did result in further revision of the state 
frameworks, but the reality of politics could not usually accommodate the reviewers’ 
criticisms. The lesson may be that, unless invited by the states, federally sponsored 
quality reviews, even by an outside group, are better at assessing the state of the field 
rather than the state of individual states. 

Finally, in meeting the quality review’s first purpose, to assess the overall quality of 
the project’s curriculum frameworks, the reviewers argued against mechanistic and linear 
models of reform. There is a parallel lesson that comes out of the quality review process 
itself Rather than produce a systematic and replicable process to assess quality, the 
review mostly succeeded in sparking the intellectual curiosity of the participants. The 
time for an investigation into issues of quality and frameworks was right, and the mix of 
perspectives and personal styles fit together well. The key lesson is that, given time for 
study and reflection, much can be accomplished and learned, not that a system for 
determining quality has been established. 
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V. CURRICULUM FRAMEWORKS AND STATE POLICY 



In this chapter, we examine the influence of the frameworks on state policy. As we 
will illustrate, the State Curriculum Frameworks Projects corresponded with a period of 
active policy development in the states. Besides developing frameworks or standards 
documents, the states began developing and piloting new assessment systems, addressing 
the issue of teacher licensure, and reconsidering a host of other policies. In some cases, 
these efforts predated the projects. 

Did the states respond to the solicitation because they were trying to implement 
reform or did they try to implement reform because they applied for and received a grant? 
The evidence suggests that the states that received grants were already trying to 
implement reform, but that in the vast majority of these states the framework projects 
were a resource for policy reform. In this chapter, we describe how and where this 
occurred in the project states, focusing on two key policy areas where the states were 
especially active: assessment and teacher licensure. 

The Influence of the Curriculum Frameworks Projects on Assessment Policy 

In the Second Interim Report (Humphrey et al., 1996b), we argued that states were 
only beginning to come to grips with the operational meaning of aligning education 
policies with frameworks. By the spring of 1997, policy alignment of the entire 
education system with high standards in the project states was still a work in progress. In 
no area of policy-making was this more true than in the reform of state assessment 
systems. 

All but one of the 16 project states were planning, developing, piloting, or 
implementing new statewide assessment systems. This high level of activity reflected the 
states’ recognition that their current assessment systems did not measure the kinds of 
student performance expected by emerging national and state standards (Laguarda, 
Breckenridge, Hightower, & Adelman, 1994). In 10 of the 16 states, the projects’ 
frameworks played a role in the assessment development process (see Exhibit 8).® Four 



® Until recently, Delaware seemed to be well on its way to aligning its educational policies with its Board- 
approved curriculum frameworks. A new assessment system was under development and a new 
professional development initiative in place. However, the governor and his new secretary of education 
recently halted work on the assessment system. Educators’ attention has turned to the coming impact of 
the end of the court-ordered desegregation plan and a statewide choice program. Still, the state’s 
frameworks continue to be used to guide revision of district curriculum. It is too early to tell whether 
standards-based reform will reemerge at the top of the state’s policy agenda. 
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of the states were not using curriculum frameworks from the projects to develop new 
assessment systems. In Nebraska, strong local-control traditions have thus far precluded 
the development of a statewide assessment system. In the District of Columbia, 
assessment development is on hold for political and financial reasons. 



Exhibit 8 

THE USE OF FRAMEWORKS IN THE DEVELOPMENT OF STATE ASSESSMENTS 



state 


Status of New 
Assessments 


Framework Used to 
Develop? 


Alaska 


Developing 


Yes 


Arizona 


Developing 


No 


Arkansas 


Piloting 


Yes 


Delaware 


On hold 


Yes 


District of 
Columbia 


Planning (on hold) 


No 


Florida 


No statewide assessment 
in science 


No 


Louisiana 


Developing 


Yes 


Maine 


Developing 


Yes 


Massachusetts 


Piloting 


Yes 


Michigan 


Revising 


Yes 


Nebraska 


No statewide assessment 


No 


New Jersey 


Developing 


Yes 


New York 


Developing 


Yes 


Oregon 


Piloting 


No 


Rhode Island 


Mathematics test in place 


Yes 


Wisconsin 


Developing 


No 



In some states, the frameworks play a particularly important role in the revamping 
of state assessment systems. In New York, the commissioner has led the state in the 
direction of high standards by requiring all students to take the challenging Regents 
Exams. Those tests and the rest of the state assessment system are being revised to 
reflect the Learning Standards. The mathematics, science, and technology section of the 
Learning Standards was developed through the project. 
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The state’s curriculum frameworks in Massachusetts are designed to support and 
guide state policy changes called for in the 1992 Education Reform Act. The math and 
science frameworks were used as models for the development of frameworks in other 
disciplines. A new state test (the Massachusetts Comprehensive Assessment System) is 
being written and will be aligned with the frameworks. Similarly, Alaska has embarked 
on an effort to align its state policies with standards for all disciplines. The framework 
grant was received after standards development was under way, and the framework 
document incorporates the standards. These standards, in turn, are informing the 
development of a statewide assessment system. 

Exhibit 8 also illustrates that in 6 of the 1 6 states, policy-makers did not use the 
federally funded frameworks to develop new assessments. However, four of those six 
states are actively planning or developing new assessment systems. Clearly, having a 
completed framework project was not a necessary criterion for developing a new state 
assessment system. Although some states did systematically integrate the project’s 
frameworks into broader state reforms, these four states worked to revise their assessment 
systems without the use of curriculum framework developed through the projects. The 
disconnection between the framework projects and assessment development can be traced 
to the difficulties of developing assessments in local-control states, the development of 
new frameworks in two states, and political shifts at the state level. 

The Limits of Policy Reform in Local-Control States 

One state has a strong tradition of local control that has thus far precluded the 
development of statewide assessments. Although the model of systemic reform makes a 
virtue out of state policy alignment, not all states have education governance systems that 
lend themselves to such an approach. In Nebraska, the state plays a modest policy role in 
education. Most Nebraskans see no virtue in state assessment systems or intrusions from 
the state capitol in Lincoln. Here the frameworks serve strictly instructional purposes. 
But even in Nebraska, there is some evidence of use of the framework by local districts 
and higher education institutions. In the three districts we visited, local curriculum 
guides were revised to match the state mathematics and science framework. State 
officials reported that this practice was common across the state. State institutions of 
higher education are reportedly using the framework document in their teacher education 
programs. In addition, the University of Nebraska at Lincoln increased the number of 
mathematics and science courses needed for admission, thereby dramatically changing 
the course-taking patterns in the state. 
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Replacing the Projects’ Frameworks 



Two of the states, Florida and Louisiana, produced framework documents through 
the federal grants but subsequently decided to replace or revise them. Louisiana 
responded to criticism of the quality of its draft frameworks in mathematics and science 
by making significant revisions. The revisions coincided with the election of a new 
governor with a standards-based reform education agenda and the vigorous support of the 
business community. As a result, the new draft frameworks enjoyed greater visibility and 
will guide the writing, pilot testing, and implementation of a new state assessment 
system. 

Similarly, Florida originally developed a framework document in science that was 
designed to move the field “a little bit.” Near the end of the grant period, the 
commissioner decided to develop a new set of standards and frameworks for all of the 
disciplines. The project’s development process became the model for the development of 
frameworks in other disciplines, and the head of the framework project was charged with 
directing the larger framework development process. Again, the new set of standards and 
frameworks will guide development of a new assessment system, although no science 
assessment is planned at this time. 

Political Changes 

The primary explanation for four states’ not using the projects’ frameworks to 
revise their assessment systems has to do with political changes that disrupted the 
projects. In Arizona, Wisconsin, Oregon, and the District of Columbia, changes in 
political leadership or changes in leadership’s educational agenda left the projects’ 
frameworks unwanted, unnecessary, or out of sync. 

For example, the political and organizational instability of Washington, DC, and its 
school system has posed a continuous challenge to the framework project. The DC 
Control Board stripped the elected school board of its decision-making authority and 
replaced the superintendent with a retired Army general. The new superintendent has 
focused on management issues, leaving the project marginalized. As one informant 
reported: “Morale in the system has been horrible for a long time . . . changes in the 
leadership [are constant] . . . and reform and changes [in practice] are tough anyway.” 

Arizona offers an even more compelling story of the influence of political change, 
as we describe in Exhibit 9. 
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Exhibit 9 

THE ARIZONA CURRICULUM FRAMEWORKS PROJECT 



The Arizona State Curriculum Frameworks Project got off to a slow start once its 
proposal was approved in the fall of 1993. At first, the state department of education 
experienced staff changes and had a difficult time hiring qualified staff to run the 
project. By September 1994, the project undenwent peer review. The reviewers 
commended the project’s plan, but saw the need for more staff time to be dedicated 
to the project. 

Throughout the fall of 1 994, committees of teachers, university faculty, and 
curriculum specialists were formed and began work on revising the state’s Essential 
Skills to reflect NCTM Standards and the emerging National Science Education 
Standards. The November 1994 elections resulted in the election of a new state 
superintendent of schools, who quickly made major changes in the Arizona 
Department of Education. With the superintendent’s help, the state initiated the 
nation’s most ambitious charter school program. 

Once in office, the new superintendent ordered a halt to the revision of the 
Essential Skills and downsized the department. The social studies framework project 
was halted, and the federal funds supporting that project were returned. Despite 
these changes, some work on the mathematics and science frameworks project did 
continue. However, the project leadership changed. Those in the department 
associated with the project hoped that things would settle down and work could 
resume. But by the late fall of 1995, it became clear that the project was at an end. 



In Wisconsin, the governor’s attempt to close the state department of education and 
the eventual downsizing of the department ultimately resulted in the framework project’s 
moving out of state. With approval from Wisconsin officials, the project is being 
completed at the offices of the Midwest Regional Consortium. There are modest plans 
for the dissemination of these products in Wisconsin. 

The Oregon case is slightly different. Oregon’s framework project’s product was 
recently released, after a review by state officials who were concerned that it did not 
reflect the 1995 revisions to the 1992 Oregon Education Act. The state is piloting new 
assessments that students must pass to earn Certificates of Initial and Advanced Mastery, 
although the project’s product has not been used to guide those new assessments. We 
describe the Oregon case in more detail in Exhibit 10. 
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Exhibit 10 

OREGON’S STRUGGLE FOR STANDARDS 



In accordance with the provisions of the Oregon Education Act (OEA), the state 
department of education created the Common Curriculum Goals (CCG), which the 
state board adopted in October 1 996. The CCG “describe the comprehensive K-1 2 
curriculum required in all districts”; however, separate statements, the Content 
Standards, describe what the state will assess. Most Content Standards are further 
specified at grades 3, 5, 8, and 1 0 by Benchmarks, statements that “establish what 
students must know and be able to do on state tests and classroom assignments.” 
The Content Standards and the Benchmarks do not cover all that is entailed in the 
Common Curriculum Goals, calling into question the motivation for teaching beyond 
what will be examined on the state assessments. 

Students demonstrating proficiency in the Content Standards will attain the 
Certificate of Initial Mastery (CIM), a grade 10-equivalent credential. Students will 
then work to achieve the grade 12-equivalent credential, the Certificate of Advanced 
Mastery (CAM). Although the state board adopted the CAM in March 1997, the state 
department is equivocal about whether students must complete the CAM within an 
“endorsement area,” an area of concentration that supposedly will provide students 
with some training in and knowledge of an occupational cluster, such as business and 
management, natural resources systems, industrial and engineering systems, or 
health services. Skeptics point out that students have little motivation to strive for the 
CIM and CAM if their high school diplomas are sufficient for college admission. 

While the state department has been defining the CIM and CAM requirements, 
the Oregon State System of Higher Education (OSSHE) has been running on a 
parallel track in defining admission standards for state colleges. The Proficiency- 
based Admission Standards System (PASS) Project defines what students entering 
the higher education system must demonstrate they know and are able to do through 
a system of multiple-choice assessments, performance-based assessments, and 
“teacher-verified” work. The PASS standards overlap the CIM in six content areas, in 
addition to defining another nine process proficiencies, only three of which overlap 
with the state’s career-related learning standards. The PASS assessment system 
may contain the same components as those of the CAM, but they are not identical. 



As our examination of the project states’ efforts to develop new assessment systems 
makes clear, assessment development often proceeded with or without the use of the 
projects’ frameworks. But, regardless of the frameworks’ role in the assessment 
development process, all states faced a series of often confounding issues associated with 
new statewide assessment systems. 
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Additional Challenges in the Development and Implementation of New 
Assessment Systems 

Beyond the technical complications of developing reliable and valid assessment 
systems, states soon discovered that they could not create assessments that would cover 
all the components of their standards and frameworks. To do so would require far more 
testing time than is thought reasonable. States have been forced to make compromises, as 
was the case with Oregon. Faced with this dilemma, Oregon selected some of the strands 
of the state’s Common Curriculum Goals (CCG) and called them Content Standards. The 
selection process meant that some of the Goals, like science and technology and science 
in personal and social perspectives, would not be assessed. 

In some states, the problem of coverage includes entire disciplines. For example, 
Florida and Alaska have new assessment systems under development that do not include 
any testing in science. Some state officials worry that teachers, particularly at the 
elementary level, will deemphasize science in their classrooms. 

The student performance on the new assessments also created challenges for the 
states. Pilots of new assessment systems in Oregon and Arkansas resulted in failure by 
large numbers of students to meet proficiency levels, particularly in mathematics. The 
states’ political leadership quickly encountered strong public concerns about the tests, 
particularly on high-stakes examinations. In Arkansas, there has been some delay in 
implementing the new assessments, in part because of concerns over large-scale failure. 
(Only 5 percent of African-American and 20 percent of white students reached 
proficiency on the pilot of Arkansas’ new mathematics assessment.) In Oregon, the 
assessment schedule has been maintained, but 1995 changes to the 1992 Oregon 
Education Act allow students to receive a high school diploma even if they do not earn a 
Certificate of Initial Mastery. In Delaware, the governor and the new state superintendent 
have halted further implementation of the new assessment system, apparently out of 
concern over the likelihood that large numbers of students would fail the high-stakes 
exams. 

Public concerns over the disproportionate number of minority students failing to 
achieve proficiency on pilots of new state assessments have also begun to surface. In 
Portland, Oregon, a coalition of minority groups threatened a boycott of the schools 
following the publication of the results of recent test scores. The group demanded that 
the school district take specific steps to close the achievement gap, including holding 
school principals accountable for student scores. A temporary compromise has been 
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reached, but it remains to be seen if Portland teachers are currently equipped with the 
skills, techniques, and resources to avert disappointing gaps in performance on the new 
assessments. 

Despite these challenges, new assessment systems may prove to be a key lever for 
reform. Much will depend on the state-specific circumstances, such as the availability of 
help for teachers, student motivation, political risks, and quality of the assessment 
system. Most states are still developing these new assessment systems, so it is too early 
to tell how the story will turn out. But judging by the level of anxiety and skepticism 
expressed by many teachers we have interviewed over the past 3 years and the 
complexity of the challenges we have identified, the implementation of the new 
assessments will not be easy. 

Aligning .Teacher Licensure Policies with the Frameworks 

Teacher licensure is another key policy area in which the project states have been 
very active. Eight of the 16 project states are developing new certification requirements, 
have them under consideration by the appropriate policy-making body, or have them in 
place (Exhibit 11). In addition, half of the project states have new recertification 
requirements in development, under consideration, or in place. Six states used their 
curriculum frameworks to guide new teacher licensure policies. 

As Exhibit 1 1 illustrates, nine states have been actively developing and 
implementing new teacher licensure policies that reflect the vision embodied in 
standards-based reform. In the New Jersey case, the reform effort represents a significant 
shift in the state’s traditional approach. When New Jersey’s project began and during 
most of the grant period, there was no consideration of recertification policies. New 
Jersey’s framework project proposal purposely excluded development of guidelines for 
recertification, given the state’s long tradition of having no policies in this area. But 
recently. New Jersey’s commissioner developed a proposal for recertification rules as part 
of the state’s push for standards-based reform. The recertification proposal is tied to the 
new state standards and frameworks, and creates a comprehensive statewide professional 
development plan. The recertification proposal would have outside groups conduct 
professional development for the state’s teachers. 
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Exhibit 11 

NEW TEACHER LiCENSURE POLiCiES 



state 


Certification 


Recertification 


Used Frameworks in 
Deveiopment? 


Alaska 


Considering 


Considering 


YesA'es 


Arizona 


None 


None 


- 


Arkansas 


Considering 


Considering 


YesA'es 


Delaware 


Developing 


Developing 


YesA'es 


District of Columbia 


None 


None 


- 


Florida 


None 


None 


- 


Louisiana 


None 


None 


- 


Maine 


Developing 


Developing 


YesA'es 


Massachusetts 


None 


In place 


No 


Michigan 


None 


None 


- 


Nebraska 


Guidelines 

completed 


None 


No 


New Jersey 


Developing 


Developing 


YesA'es 


New York 


In place 


Considering 


NoA'es 


Oregon 


In place 


In place 


No/No 


Rhode Island 


None 


None 


- 


Wisconsin 


None 


None 


- 



In New York, recertification was considered by the Advisory Council for 
Certification of School Professionals. When the Council recommended that no new 
recertification requirements be instituted, the commissioner rejected the recommendation 
and turned to his Task Force on Teaching to make a different recommendation. The 
commissioner insists that the state’s new Learning Standards require a system of ongoing 
professional development. Some state officials speculate that the state will require 
recertification but leave the details to the districts. 

In Maine, the legislatively established committee to reform teacher preparation, 
certification, and continuing professional development policies took a long time getting 
organized. Between 1996 and 1997, the committee did hold a few meetings, but it was 
only recently that leadership from Maine’s Statewide Systemic Initiative and State 
Curriculum Framework Project were invited to join the committee. The addition of the 
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SSI and SCF Project leadership to the committee helps ensure that the framework will 
help guide the changes in teacher licensure. The committee was beginning to look at an 
outcomes-based teacher certification process that relied on more than automatic 
certification by virtue of completing an approved teacher education program. 

Arkansas’ legislatively mandated Task Force on Teacher Licensure was charged 
with making recommendations for changes in the way teachers and administrators are 
prepared in the state. The Task Force was established by Act 236 and is charged with 
reshaping the whole system of licensure and recertification. Although still under 
consideration, the Task Force’s work was expected to establish middle school 
certification, a more integrated science credential, and ongoing professional development 
requirements. 

As was the case with state efforts to develop new assessments, some states 
developed new teacher licensure policies without using the projects’ frameworks. 

Oregon stands out as a state that continues to actively revise its state policies without the 
project’s assistance. The new rules established authorizations at the early childhood, 
elementary, middle, and high school levels. Teachers earn an “Initial Teaching License” 
by completing an approved course of study and demonstrating their competency in five 
areas: 

• Plan developmentally appropriate instruction 

• Establish a classroom climate conducive to learning 

• Engage students 

• Evaluate, report, and use information on student progress 

• Exhibit professional behaviors. 

First-year teachers are assigned to a mentor. After 3 years and the acquisition of a 
master’s degree and an approved professional development program, teachers are eligible 
for a “Continuing Teaching License.” Thereafter, teachers are required to update a 
professional development plan annually. The lack of resources for professional 
development, however, could seriously weaken the state’s new recertification rules. 
Successive tax-limitation initiatives have left both the state and the districts without many 
resources to devote to professional development. In addition, there are questions about 
the state department of education’s capacity to guarantee that the professional 
development offerings will reflect the state standards and the goals of the state reform 
effort. 
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All states face similar challenges regarding adequate resources for professional 
development. And, like Oregon, all the states face questions about capacity. 
Massachusetts serves as a case in point. Massachusetts’ widely heralded recertification 
requirements, which were part of the 1992 reform act, proved to have less-than-hoped-for 
results. Although teachers are now required to acquire continuing professional 
development points, the state department lacks the capacity to carefully monitor what 
counts as professional development. In addition, the requirements for completion of 120 
professional development points every 5 years resulted in an initial flurry of professional 
development activity, a drop-off of activity after the first few years, and a lack of quality 
control. State Board of Education Chair John Silber charged that teachers could get 
points by going on a cruise. In addition, there is a misalignment with the state’s science 
framework, which pushes for an integration of the disciplines, and recertification, which 
requires teachers to be certified in one discipline. The state department understands these 
problems and may have an opportunity to strengthen the program in the near future. 

Conclusion 

As this chapter demonstrates, the framework project states — like nearly all states — 
have been extremely active in revamping their policy systems. In particular, there is now 
unanimous attention to standards for student learning. Exactly how far states have 
progressed, the specific policies they have paid attention to, and the degree to which the 
federally funded framework projects have been used in policy development varied widely 
and depended on a set of state contextual factors. 

In 10 of the 16 project states, the Eisenhower State Curriculum Frameworks 
Projects have played a useful role in helping policy-makers to shape new assessment 
systems. Half of the project states used the frameworks in the development of new 
teacher licensure policies. In addition, there is evidence that the projects had some 
influence in other policy areas. For example, Florida’s project became the model for the 
development of a new generation of frameworks, which promise to be tied to the new 
state assessment system. The Massachusetts project’s use of study groups in mathematics 
and science has been expanded to other disciplines. And, as we discuss at length in the 
next chapter, Maine has focused all of the state’s professional development offerings on 
th© implementation of its standards and frameworks. 

As a result of all of these findings, it is safe to say that the glass is more than half 
full when it comes to the frameworks’ influence on state policy. However, in all states, at 
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least two factors tended to limit that influence. First, as the name implies, the State 
Curriculum Frameworks Projects were viewed by some state officials as projects, just 
like himdreds of other projects they oversaw. For many long-time state department 
officials, district administrators, and overburdened teachers, a project is something that 
ends, rather than something to serve as an enduring guide for all state policies. Second, 
even in states where project leadership thought of the project as part of a grand scheme 
for changing policy, there was no systematic plan for changing policy. The lack, of a 
comprehensive plan for changing policies is explained partially by the fact that project 
leadership was typically not well positioned in the state education department to 
influence high-level decisions. In addition, project leadership usually had expertise in 
professional development rather than policy-making. 

Despite the lack of a systematic plan for changing policy, the majority of 
frameworks developed through the projects were a useful resource in the states’ efforts to 
make education policy align with the goal of high standards for all students. In that 
sense, the projects demonstrated the influence that federal policy can have at the state 
level. It is important to note, however, that in each state it was important that the 
framework development and aligned efforts were seen as state-driven, not as federal 
imperatives. 

Finally, the project states showed that standards can bring about some measure of 
policy coherence. The overall level of activity under the standards baimer is impressive, 
and curriculum frameworks were an important piece of standards-based reform efforts in 
the majority of project states. But even the most coherent set of policies can stand in 
splendid isolation if they are not accompanied by significant and sustained opportimities 
for teachers and administrators to incorporate the intent of the policies into their practice. 
In the next chapter, we turn to an examination of the project states’ efforts to help 
educators use frameworks to improve their practice. 
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VI. IMPLEMENTATION STRATEGIES FOR IMPROVING 
TEACHER PRACTICE 



In the preceding chapter, we argued that the framework projects have been used to 
varying degrees to guide state policy-making in assessment, teacher licensure, and other 
areas. The influence of framework projects on state policy has been in evidence in a 
majority of states, but the frameworks are typically resources for policy-makers rather 
than drivers of broader changes in state policy. In fact, the projects devoted more time 
and attention to introducing teachers to the framework documents through professional 
development activities than to aligning state policy with the frameworks. In this chapter, 
we shift our focus to the projects’ strategies for using the frameworks to improve teacher 
practice. 

Given the often changing political environments in which the projects operated and 
the myriad influences on teachers’ practices — such as prior knowledge, educational 
background, experience, available resources, and parents’ demands (Grant, Peterson, & 
Shojgreen-Downer, 1996) — the task of introducing curriculum frameworks as a vehicle 
to change instruction was far from easy. To succeed, projects had to tackle a series of 
challenges: 

• How to create opportunities and incentives for policy-makers, administrators, 
parents, and teachers to “pop the shrink wrap” and examine the documents. 

• How to promote a deep understanding of the framework contents. 

• Given limited resources, how to balance the need to reach large numbers of 
teachers with the need to provide in-depth professional development on how to 
effectively implement the frameworks. 

• Finally, as the grant ended, how to sustain long-term commitment to the use of 
frameworks as guiding documents. 

Throughout this discussion, it is important to keep in mind that, in many ways, this 
notion of implementation is not as discrete or linear as it may appear to be. For most 
states, the curriculum frameworks were not developed and then implemented in an 
orderly sequence of events. Many states would argue that the projects had no clear line 
between implementation and development. For example, many projects involved a wide 
range of stakeholders in the development and review of frameworks to promote buy-in 
and awareness of the ideas and content. This is particularly true of the approaches taken 
by projects in New York, Florida, Massachusetts, and Nebraska. In addition, the 




59 



73 



projects’ implementation strategies continued to evolve and were enacted well after the 
grant period ended. 

Implementation Strategies 

As discussed in past reports, states took varied approaches to disseminating the 
frameworks — in some cases sending copies to every teacher in the state (see Humphrey et 
al., 1996a). In the hope of preventing an unopened box of frameworks from landing in a 
district warehouse (as it did in one district we visited), projects developed multifaceted 
strategies to ensure that the frameworks reached the receptive hands of their intended 
audiences. Exhibit 12 illustrates the range of approaches taken by states to encourage the 
examination and implementation of framework documents. The subsequent section will 
explore each strategy in more detail. 

Exhibit 12 

CURRiCULUM FRAMEWORK iMPLEMENTATiON STRATEGiES* 





Professional 
Development 
and Technical 
Assistance 


Study 

Groups 


Resource 

Guides 


Pilot 

Sites 


Technology 


Alaska 


/ 






✓ 




Arizona 












Arkansas 


/ 










Delaware 


/ 




/ 




/ 


District of 
Columbia 


/ 




/ 






Florida 


/ 






/ 


/ 


Louisiana 


/ 




/ 


/ 


/ 


Maine 


/ 


/ 


/ 


/ 


/ 


Massachusetts 


/ 








/ 


Michigan 


/ 




/ 




/ 


Nebraska 


/ 








/ 


New Jersey 


/ 








/ 


New York 


/ 








/ 


Oregon 










/ 


Rhode Island 


/ 








/ 


Wisconsin 













*Our discussion of these strategies is not limited to project-supported activities. 
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Professional Development and Technical Assistance 

The implementation strategy used most frequently in the 16 states was professional 
development and technical assistance associated with curriculum frameworks. Not all of 
these professional development opportunities were paid for with project funds, as the 
states made efforts to leverage resources to support the implementation process. The type 
of opportunities that states offered varied across and within states on a number of 
dimensions, including content, intensity, and duration. 

The majority of states employed several different types of professional development 
and technical assistance activities at the same time. For example, Alaska’s project 
developed a two-pronged approach to professional development. First, the project tapped 
into existing professional development efforts of the state’s Math and Science Consortia, 
which covered approximately half of all districts. Project staff worked to infuse 
framework-related content into the Consortia’s 3-week summer institutes and 2-day 
advanced institutes. To reach the non-Consortia-affiliated districts, the project developed 
two distance learning courses, one targeted at curriculum committees and the other at 
teachers and administrators. These courses were an orientation to the frameworks and an 
introduction to the content, instruction, and assessment methods recommended by the 
frameworks. 

Nebraska employed at least three concurrent efforts. Mindful of the local-control 
tradition in the state, teachers were used as the trainers and presenters of frameworks 
across the state. Teachers attending the framework workshops received copies of the 
framework at the workshops, followed by 2 to 7 hours of training. At the same time, the 
state math and science coordinators were available to provide inservice professional 
development on the frameworks to districts. In addition, the frameworks were used in 
teacher preparation programs at the state universities and colleges. 

In Florida, the framework project funded six trainers (1/3 time each) to conduct 
staff development around the state. On one end of the scale, trainers conducted 
“awareness” sessions at teacher and district conferences and institutes. On the more 
intensive end, trainers customized workshops for curriculum developers and teachers to 
meet the demands of regions. For example, in the faster-paced southern part of the state, 
with many educators to reach, trainers offered 1- to 2-day workshops — allowing them to 
reach more than 700 people in a 7-week period. In the less densely populated 
northeastern part of the state, trainers conducted 4-day workshops. All workshops 
included hands-on activities that introduced and helped educators leam to use the science 
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framework and the Curriculum Planning Tool. In addition, the state department of 
education established Area Centers for Educational Enhancement to provide districts with 
staff development and assistance related to frameworks in all disciplines. Because the 
state is developing assessments in mathematics and reading but not science, recent 
activity in the Area Centers has not focused on science (the content area funded by the 
Eisenhower SCF grant). 

Delaware’s Elementary Science Project involved 63 elementary schools in nine 
school districts. Leaders at each school received 30 hours of training per year, as well as 
5 additional days of training. Training was linked to existing curriculum units in kits 
from the Smithsonian’s National Science Resource Center. Sessions focused on science 
content, teaching strategies, assessment, and classroom management, as well as the 
curriculum materials in the kits and standards. The lead teachers were expected to follow 
up and assist teachers in their schools with similar curriculum activities. In 1995-96, 
approximately 350 teachers participated, and in the 1996-97 school year, another 900 
were involved. Delaware also delivered professional development to high school 
teachers in its schools via the Science Van Project. In 1996-97, 32 teachers of biology 
and chemistry in 17 high schools participated. Paid for by private-sector funds, the van 
brought equipment, materials, and science teacher leaders to the schools to demonstrate a 
lesson and work with each teacher for 2 to 3 days. 

Similar descriptions could be made of most of the other projects, but a few states 
had particularly ambitious and exemplary professional strategies. Maine’s professional 
development strategy worked well because it linked to other reform efforts and resources 
in the state. As a small state, Maine had significant advantages over large states like New 
York. However, its approach warrants the more detailed description we present in 
Exhibit 13. 
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Exhibit 13 



PROFESSIONAL DEVELOPMENT IN SUPPORT OF STANDARDS-BASED 

REFORM IN MAINE 



Maine has dedicated much of the state’s professional development resources to 
helping teachers understand and use the mathematics and science curriculum 
framework. The state’s multifaceted plan represented a coordinated effort, garnering 
funds from a variety of federal, state, and regional sources. 

A key roll-out vehicle for Maine included 2-day Problem Solving in Science and 
Math (PRISM) conferences. Co-sponsored by the state department of education, the 
Eisenhower state program, the state math and science teachers’ associations, and 
the state SSI, these conferences are held three or four times per year and enroll 
approximately 300 to 400 teachers each. Organized for grade-level clusters, the 
conferences contained interactive introductory workshops on the framework, as well 
as more advanced sessions for the 1/2 to 1/3 of teachers who had previous exposure 
to the framework. 

During one PRISM workshop that we attended, on using frameworks to guide 
assessment and instruction, workshop leaders spent time on very precise analysis of 
classroom assessment approaches, making explicit links to specific performance 
indicators from the framework. Workshop leaders also offered an introduction to the 
development of rubrics for classroom use. In another session, teachers were 
introduced to the frameworks, participated in a series of hands-on activities related to 
specific content standards, and broke into groups to discuss key questions, including 
the relevance of standards, how the critical-skills model supports the use of 
standards, and questions related to classroom implementation. Teachers we spoke 
to after the session seemed enthusiastic, although concerned about the amount of 
time they will need to incorporate these new practices into their teaching. 

In addition to the PRISM conferences, two sources provided districts with 
technical assistance related to the frameworks: (1) fourteen SSI specialists provided 
a range of assistance, from introduction to the frameworks, to mapping local 
curriculum and textbooks into the frameworks, to linking the frameworks with student 
assessment: and (2) the state department of education’s Regional Education Support 
Teams (RESTs) devoted 50 days to math and science annually and helped local 
educators learn to use the frameworks. Both groups held a joint training in the 
summer of 1996 to ensure that they infused similar content and approaches into their 
technical assistance efforts. 
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Exhibit 13 (Conciuded) 



The framework project also tapped into existing regional networks of educational 
leaders to provide facilitated “courses” on the frameworks. Educators in these 
courses were later expected to participate in leadership networks within districts. 
Lasting either 2 full days with two after-school follow-up sessions or six after-school 
sessions, each course involved a team of 12 to 40 district teachers. Nearly every 
district in the southern region of the state took advantage of this opportunity. With 
assistance from SSI Beacon facilitators, teams were expected to (1) know the content 
of the framework and how it could influence curriculum, instruction, and assessment; 
(2) become part of a district-level leadership network that planned and shared 
strategies for using the framework; and (3) put together a portfolio describing school 
and/or district projects. 

At the beginning of its framework project, Maine selected six schools to serve as 
pilot sites (five were funded by the Eisenhower grant and one by the Regional 
Consortium). The schools represented a range of stages in the mathematics and 
science reform process, and each received a 1-year grant of $10,000 to cover costs 
of release time, professional development materials and consultants, and/or travel to 
workshops and meetings. Although some made more progress than others, and 
many experienced difficulties, the endeavor was considered fairly successful, yielding 
instructive lessons for others. 

The state prepared a case study of each site that will be included in an 
Instructional Resource Manual. It is hoped that the case studies will serve as a “how 
to” guide for other schools trying to implement the framework. Besides the case 
studies of the pilot schools, the manual will include curriculum units developed by 
K-12 math and science teachers that list outcomes, instructional strategies, and 
sample assessments. Maine also used SSI-funded Beacon Centers in the roll-out of 
the framework. Like the pilot sites. Beacon schools experimented with and chronicled 
their experiences with framework- and standards-based reform. 

Much of the professional development activity in support of the frameworks in 
Maine was a direct result of the close links that the project established with other 
reform initiatives. Notably, Maine established an advisory board that served both the 
framework project and the state’s SSI. In addition to close ties to the SSI, Maine 
used local Goals 2000 grants to support technical assistance on use of the 
framework. Goals 2000 funds also allowed districts to continue work started with 
frameworks project funds. For example, one project-sponsored pilot site received a 
$25,000 Goals 2000 grant that allowed it to build on its previous year’s efforts to 
implement districtwide change based on frameworks and standards. We also found 
examples of professional development opportunities in support of the frameworks that 
were partly funded by Title I, state Eisenhower Professional Development funds, 
NYNEX, and the Regional Alliance. 
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The Maine example suggests two components of an effective implementation 
strategy that were in evidence in some of the other states. Project leaders recognized the 
need for additional support materials, realizing that the curriculum framework lacked 
enough direction or examples for teachers. In addition, project leaders were particularly 
successful in using other professional development resources to help introduce the 
framework to teachers. We turn to examples of these two strategies in other states in the 
next two sections. 

Resource Guides 

As was the case in Maine, other states developed or are developing documents 
designed to accompany curriculum frameworks and state standards. In Michigan, a 
Science Education Guidebook was developed with project funds to translate the well- 
established state science curriculum framework into classroom practice.’ Developers 
hoped that this guidebook would serve as a resource to teachers unable to attend training 
opportunities. The document covers a wide array of topics: highlights of links between 
the guidebook and the framework; practical suggestions for planning science programs, 
including sample plans, scope and sequence charts, and additional resources; information 
on teaching strategies from Science for All Americans', highlights of the state assessment 
program; and synopses of instructional modules from state and national sources. 
Michigan project staff also developed the “Connecting with the Learner” toolkit, which 
was released in 1 997. The toolkit was designed to help curriculum developers and 
workshop facilitators integrate equity issues into the curriculum. This resource consists 
of 50 activities divided into 6 categories: curriculum, instructional strategies, partnering 
with parents, learning styles, model programs, and examining beliefs. 

Similarly, Florida produced a Curriculum Plarming Tool (CPT) that includes ideas 
for developing curriculum linked to the science framework (in the last year, it was 
expanded to include all seven subjects covered by new state standards). The document 
includes example activities from districts around the state and approximately one lesson 
plan for each benchmark outlined in the state standards. Because of several obstacles, 
however, the quality of the final product was lower than had been expected. The CPT 
was originally produced on diskette and placed on the Internet, which for many educators 
was not easily accessible. Although it recently produced a limited version on CD-ROM, 
the state is still working to upgrade the version and increase production. In addition, the 



’ Given that the state had already produced a high-quality and well-respected science curriculum 
framework, the Eisenhower framework funds for science were deployed to underwrite the guidebook. 



amount of research necessary to investigate possible copyright problems reduced the 
number of activities that were ultimately included in the document. Finally, there seems 
to be confusion over the purpose of the CPT. Whereas some saw it as a compendium of 
choices with multiple lessons per benchmark, project staff argue that its purpose was to 
stimulate teachers’ development of their own lesson plans aligned with state standards. 
Nevertheless, the tool has been used widely and, apparently, teachers find it to be useful. 

New York also produced resource guides for its curriculum framework and 
standards documents. Although produced with non-Eisenhower funds, the guides 
explicitly refer to the framework and offer examples of resource and research material 
that can serve to inform local curriculum development. Available on the state department 
of education’s Web site, ihe Mathematics, Science, and Technology Resource Guide is 
divided into three parts. The first section contains elements considered essential in 
planning standards-based math, science, and technology curriculum. This includes 
strategies for integration and creating equitable learning environments, best practices, and 
samples of curriculum, scope and sequence materials, and assessments. The second 
section details experiences of classroom teachers, including example activities used to 
“bring the learning standards to life in the classroom.” The titles of some of these 
learning experiences include: “Math, Monarchs, and Metamorphosis; and Exploring 
Transformations,” “Inverse (Indirect) Machines; and Statistics of the M&M Candy,” and 
“Bill and Ted’s Eggsellent Adventure.” The final part examines assessment models, 
including examples of integrated math, science, and technology tasks and materials 
developed across the state. 

Developing and disseminating state guidance on curriculum and pedagogy remains 
a delicate balancing act in some states. In Massachusetts, concerns over offending local- 
control sentiments with a state-sponsored, seemingly prescriptive document have delayed 
the development and release of a framework implementation guide. However, 
Massachusetts has been very active in offering professional development opportunities to 
help teachers identify curriculum materials that are aligned with the frameworks. 

Technology 

Most states used technology — albeit in a limited capacity in most cases — to 
increase awareness of and access to the curriculum frameworks. Many states have placed 
the curriculum frameworks and/or other products on the Internet, including Alaska 
(reference kit), Arkansas (frameworks), Florida (Curriculum Planning Tool), 
Massachusetts (frameworks). New Jersey (frameworks). New York (frameworks), and 
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Rhode Islsnd (frameworks). Although making the documents available is the first step, 
the more critical issue is ensuring use of these resources. Unfortunately, few states have 
data indicating the number of educators who have logged onto the Internet to view these 
documents. Given the uneven technological capacity of schools across the country, it is 
fair to estimate that many teachers have not taken advantage of the electronic 
information. 

A few states have used technology to influence educators directly. Maine’s 
interactive television system was used for awareness and feedback sessions that were 
conducted live and also made available free on videotape. Two of the three major 
professional development efforts in Alaska used distance learning and/or videotapes. For 
example, a teacher and administrator professional development course occurred via one- 
way video and two-way sound. Organizers planned to deliver this course on video by 
mail and some type of electronic or audio correspondence with instructors at the 
University of Alaska Southeast. 

Several states’ experiments with technology, although partly successful, uncovered 
the limits of this medium. For example, Florida struggled to produce its resource tool on 
CD-ROM, eventually producing a limited version. Oregon’s product, a CD-ROM 
compilation of 14 district projects, encountered similar limitations. Not anticipating the 
rapid changes in technology, the project invested in a presentation format that is now out 
of date. The final product failed to capture the full breadth of the local experience, since 
the majority of information was presented in text form. Photos were included, but the 
lack of motion video clearly dated the technology. 

Local Sites 

Like Maine, a few states chose to pilot frameworks in local sites. This strategy was 
designed to have pilot sites serve as examples for other sites attempting to translate 
frameworks into actual curriculum and instruction. This strategy was working well in 
Maine, but other states that sponsored pilot districts or schools had mixed results. 

For example, Florida reportedly piloted the science framework in 35 SSI-funded 
model schools. The model schools were to provide a place where people could go and 
see working programs. The strategy established strong ties between the framework 
project and the SSI, but the Florida SSI was defunded by NSF. New York plaimed to 
pilot the new Learning Standards in the 10 SSI Research and Demonstration schools that 
were the focus of its SSI, but not all the schools paid attention to the Learning Standards. 
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Arkansas’ attempt to use local sites as “beacons” of reform did not go well. The 
project selected six pilot districts to develop mpdel curriculum guides based on the state 
framework. Each participating district received $8,000 for release time, substitutes, and 
stipends for participating teachers. District representatives received some training, but 
teachers we interviewed complained about the lack of adequate support. In one of the 
districts, teachers indicated that being put on the committee to develop the curriculum 
guides was a form of punishment. The pilot districts produced curriculum guides that 
failed to reflect the vision of state officials. Typically, the guides contained long lists of 
textbook page , numbers that were associated with each strand of the state curriculum 
framework. Despite the poor quality, the state distributed some of the guides to all the 
districts in the state with instructions to revise their curriculum guides in line with the 
framework. State officials estimate that 20 percent of the districts either did nothing or 
copied the curriculum guide of another district, while 25 percent of the districts took the 
task seriously. 

Oregon’s unique framework project was focused on local sites and their efforts to 
implement standards, rather than on the creation of a framework document. As we 
discussed earlier, the project’s intention was to capture each of the local site initiatives on 
CD-ROM and distribute it to all the districts in the state. The local initiatives illustrated 
some of the challenges associated with standards implementation at the local level. 
Although some local initiatives made significant contributions to the schools and 
districts, others suffered from personnel changes or a lack of leadership. Nearly all the 
local initiatives faced the challenge of being another project among dozens of projects. 

Study Groups 

One strategy that was unique to Massachusetts was the use of study groups. In 
Massachusetts, the framework project funded district study groups designed to engage 
teachers and administrators in discussions about how to integrate the state curriculum 
frameworks into their districts. Winning proposals entitled districts to send a 
representative to statewide training on group facilitation and receive funding for 1 year to 
examine the frameworks. 

In general, teams of 8 to 10 teachers and administrators served on study groups for 
each discipline. Going page by page through the documents, the groups discussed the 
ideas, interpreted them into matching classroom activities, and identified materials 
available to implement these activities. One Boston study group came up with themes for 
each grade level that correlated with the frameworks: teachers left the group with a 



specific activity or plan that they could share with other teachers at their grade level and 
implement in their classrooms. Several districts used study groups to develop their own 
curriculum frameworks and guides, further specifying the broader state-level documents. 
In year 2 of the grant, there were 366 study groups in math and 357 in science statewide. 
The state is replicating the study group model in its implementation strategy for the other 
disciplines’ frameworks. 

Links to Other Reform Efforts 

In all 16 states, the Curriculum Frameworks Project was one of many reform efforts 
under way. As in the Maine example, states that purposefully designed their projects to 
link with these other initiatives were best able to help teachers understand and use the 
frameworks. In states such as Maine, Massachusetts, and Michigan, many federal- and 
state-sponsored reform initiatives converged — they shared common leadership and jointly 
sponsored activities, making the pieces indistinguishable from the whole. With these 
links, projects were also well suited to sustain their efforts once the Eisenhower grants 
ended. The following section explores the integration of framework projects with other 
reform initiatives. 

Links to NSF’s Statewide Systemic initiatives 

Nine of the 16 State Curriculum Frameworks Projects were located in states 
engaged in Statewide Systemic Initiatives (SSI) funded by the National Science 
Foundation. Given that the SSIs shared the same general goals of standards-based reform 
to improve mathematics and science education, most projects forged close links to these 
initiatives. The considerable resources available to the SSIs ($10 million for 5 years) 
allowed many of the projects to expand into areas and activities that would not have been 
feasible with the Eisenhower funds alone. 

In several states, the SSI and framework projects were virtually one and the same. 
Massachusetts’s SSI, Partnerships Advancing Learning for Mathematics and Science 
(PALMS), overlapped with the framework project in terms of goals, activities, and staff 
The project director believed that the framework project allowed the state to extend 
PALMS beyond its pre-K-8 focus to include pre-K-12. The framework project fit 
perfectly into the PALMS strategic plan, which called for the creation of frameworks, 
professional development, and other activities to improve the quality of math and science 
education in the state. In addition, the SSIs continued the work of framework projects 
once the grants ended. 
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Although Nebraska’s SSI and framework projects were run by different entities, 
over time the two developed a strong cooperative relationship — co-sponsoring 
professional development workshops and other activities. Nebraska’s SSI was well under 
way in the state when the framework project began — setting the stage for reforms called 
for by the Eisenhower project. Working in tandem helped increase the visibility of the 
state’s math and science coordinators and raised awareness of the importance of math and 
science within K-12 education. Over time, Florida’s SSI and framework project also 
developed a close collaboration that included SSI-sponsored staff development activities 
focused on the framework and framework piloting in SSI “Discovery Schools.” 

The other six states that had SSIs and State Curriculum Frameworks Projects 
followed similar patterns of coordination and cooperation. Those states that housed the 
SSI and framework projects in separate offices of large state departments, like New York, 
or in completely different governmental bodies, like Arkansas, faced the biggest 
coordination challenges. Those states that ran both initiatives out of the same office, like 
Maine, or institutions, like New Jersey (where both initiatives were housed at Rutgers 
University), found coordination to be less challenging. 

SSI states that did not have State Curriculum Frameworks Projects were often 
involved in standards and framework development. The lack of a framework grant 
appears to have limited the resources that those SSI states were able to devote to 
standards and framework development. However, the success of those development 
efforts was more closely tied to the overall success of the SSI project and the state context 
rather than to the presence of a framework grant. For example, Colorado developed 
influential framework documents with the assistance of the SSI and the leadership of 
Governor Romer. On the other hand. South Dakota’s SSI developed framework 
documents, but they have yet to become central to state policy-making or teacher practice 
in the state. 

Links to Goals 2000 

Although the explicit links between the projects and state Goals 2000 activities are 
still evolving, we uncovered at least five instances in which the two initiatives worked 
collaboratively. Several states, such as Massachusetts and New York, placed stipulations 
on local Goals 2000 grants, insisting that any district receiving funds demonstrate use of 
state curriculum frameworks and alignment of activities with those frameworks (in 
Massachusetts, this stipulation applied to study group districts). 
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Other states, such as Nebraska and Delaware, are using the curriculum frameworks 
as a springboard for state standards development under Goals 2000. The Nebraska case 
also illustrates the potentially fragile relationship between the two initiatives. Nebraska 
framework staff looked on the new standards development efforts with some anxiety, 
fearing that if the new Goals 2000 standards were declared or perceived to be mandatory, 
the voluntary flavor and appeal of the antecedent frameworks would be significantly 
compromised. 

In other states, the coimection to Goals 2000 seemed to generate momentum for 
both projects. In Louisiana, Goals 2000-sponsored standards development built on and 
revised the frameworks developed through the State Curriculum Frameworks Project. At 
the same time, the linkage to Goals 2000 added visibility and credibility to standards- 
based reform promoted by the framework project. 

Links to Eisenhower State Grant Programs and Regionai Consortia 

Links between the framework projects and the Eisenhower State Grant Program 
also evolved over time. Florida state Eisenhower funds provided partial support for 
curriculum framework committee meetings. In Maine, the state’s Eisenhower director 
was also the SSI’s principal investigator, which facilitated collaboration among the 
various programs. In Massachusetts, districts receiving framework project and 
Eisenhower funds to conduct study groups needed to demonstrate that they were 
implementing the curriculum frameworks. Both New York and Nebraska used 
Eisenhower higher education funds to support workshops on the frameworks and 
standards. New York also required grant proposals from institutions of higher education 
to identify which specific standards the proposed project would address. 

Some of the Regional Consortia also provided support to the framework projects. 

In Maine, the Regional Alliance (TERC) provided $10,000 for one of the six pilot sites 
involved in testing the state framework. In Alaska, a Regional Consortium staff member 
worked closely with state department staff to provide professional development and 
technical assistance on the frameworks. In the District of Columbia, the Regional 
Consortium helped build the capacity of project leadership during the initial years, 
planning meetings of the advisory board, providing staff development for teachers who 
were to conduct training, and providing standards materials to schools. Similarly, in 
Oregon, the Northwest Regional Consortium assisted in the early development of the 
project. As we reported earlier, the Midwest Consortium for Mathematics and Science 
Education eventually became the home of the Wisconsin project. 




71 



85 



Not all of the Regional Consortia were so directly involved in the projects. 

However, some Consortia made significant contributions to states that did not have 
framework grants. For example, the WestEd Regional Consortium played a pivotal role 
in Utah’s development of a science framework. 

From Strategies to impacts 

State officials were under no illusions about the challenges of implementing the 
curriculum frameworks. As the examples of state strategies suggest, 14 of the 16 states 
employed multiple implementation strategies. The most ambitious strategies maximized 
teachers’ exposure to the documents and provided both the time and resources for 
teachers to study, reflect, and experiment. These strategies required a coordinated 
effort — one that garnered the resources of other state, federal, and local reform efforts. It 
also required that implementation strategies extend beyond the life of the grants. 

As we have emphasized earlier, the use of curriculum frameworks by policy-makers 
and educators continues to evolve. In the 16 project states, most local districts are in the 
early stages of revising their local curricula and implementing the changes. Having said 
that, standards-based reform has had some impacts at the district and school levels. In the 
next chapter, we examine those impacts in a sample of districts as we address the 
question “What happens to frameworks once they arrive at the district offices?’’ 
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VII. IMPACT IN THE DISTRICTS 



In the final year of the evaluation, our primary charge was to understand how the 
frameworks were used once they were distributed. As might be expected, there is great 
variation across the districts. In some districts, we were hard pressed to find anyone who 
was familiar with the state’s documents. Sometimes, the district had not yet gotten 
around to dealing with standards, or it was fully engaged in other pressing issues. 
Recognizing that it was still too early to expect widespread changes in policy and 
practice, we focused the data collection activities on states and districts where, on the 
basis of input from state framework personnel, we expected to find districts engaged with 
the frameworks. Thus, we purposely examined places that would give us insights into 
how framework documents were being used. 

This chapter presents cases of five school districts and three schools that illustrate 
how the frameworks are used by district officials, principals, and teachers (all district and 
school names are pseudonyms). These cases portray districts at different stages of 
reform, each with slightly different experience with state frameworks. They were 
selected from a sample of 20 districts that we visited in the 8 case study states during the 
spring of 1997. The cases represent the range of involvement with standards-based 
reform across the 20 districts. After presenting the cases, we discuss cross-cutting 
themes. The chapter concludes with a discussion of some of the broader challenges 
facing teachers as they move ahead with standards-based reform. 

Best-Case Scenarios: Use of Frameworks in Districts Well on Their Way to 
Reform 

The first two cases feature districts that appeared to have benefited greatly from the 
use of their states’ frameworks and other standards documents. Our criteria for selecting 
the best-case scenario cases included; 

• Evidence that the district or school was using the framework to guide local 
curricula and teacher practice. 

• Evidence that local curricula and teacher practice reflected the vision described 
in the state framework. 

• Evidence that district policies and school policies supported the implementation 
of the state framework. 
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® Evidence that support systems, including professional development 

opportunities, were available to teachers and administrators in their efforts to 
implement the state framework. 

• Evidence that assessments aligned with the framework were being used to 
inform teacher practice. 

The first case reflects the experience of a district that was far along the path of reform 
even before it began to use the new state curriculum frameworks. The second case 
focuses on a successful school that has been actively engaged in reform in a district that 
has more recently attempted standards-based reform. 

Gemtown, IViassachusetts: Standards-Based Reform at the District and iru 

Classrooms 

Gemtown is a small, middle-class, predominantly white manufacturing city in the 
southeastern region of Massachusetts. Its school district serves more than 6,000 students 
in six elementary schools, three middle schools, and one high school. 

Guided by a strong-minded superintendent, the Gemtown public schools engaged in 
education reform efforts before the development of state curriculum frameworks. Early 
on, the superintendent reassigned teachers and administrators to different schools to shake 
up the system, required K-8 teachers to teach the same students two years in a row, and 
established business partnerships for every school. The superintendent also promoted a 
culture of professionalism within the district. Considered a professional development 
district for preservice, Gemtown pairs student teachers with panels of master teachers 
during their preservice and provides them with coaching assistance and opportunities to 
attend workshops alongside teachers. The district employs a full-time professional 
development coordinator. In addition, all teachers must take a three-credit course each 
year to obtain salary increases. Because of its aggressive recruitment efforts, the district 
has been able to attract teachers from a top university not far from the school. 

Before the release of the state frameworks, groups of teachers and administrators 
developed district Content Standards and Learner Outcomes in each subject area for 
grades 4, 8, and 12, which reflected the national standards. Gemtown received copies of 
the frameworks just as it was beginning to implement its own district Content Standards 
and Learner Outcomes. In 1994-95, the district received support through the State 
Curriculum Frameworks Project for study groups of volunteer teachers to examine the 
state documents and compare them with the Gemtown standards. After performing this 
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comparison, the study groups offered suggestions for revisions to the standards, adding 
specificity to each grade. 

In 1995-96, district study groups developed interim standards for grades 2 through 
6. Again, group members drew from state and local standards documents. In the 1996- 
97 school year, the district used study group funds for articulation. Teachers from 
different schools and grade levels met to discuss the entire K-12 curriculum and fill in 
gaps. The science group also developed a scope and sequence for the science curriculum, 
specifying yearly themes and blocks of study for each K-8 grade. 

Given the standards- setting activity that took place before the release of state 
documents, Gemtown was well situated to move ahead with the actual implementation of 
standards-based reform. Having chosen science as the lead discipline, the superintendent 
asked all teachers and administrators to group courses according to science skills, big 
concepts, and ideas in the frameworks. To support this model, the district devoted two of 
its three professional development days to the science curriculum framework. In 
addition, many teachers took advantage of the PALMS (the state’s SSI) professional 
development opportunities. 

Thus, although frameworks did not introduce radically new information to the 
district, they clearly helped to validate the reforms the district had already started to 
implement. The frameworks also helped to unify teachers and administrators around a 
common vision, particularly in science education. On the basis of interviews with five 
teachers and a focus group with six teachers, we had the impression that Gemtown 
teachers were generally pleased with the direction of the district and the attention being 
paid to curriculum. 

• One teacher said it has pushed him and his colleagues to think about and discuss 
issues of math and science, that it has raised the importance of examining 
curriculum and constantly revising it. 

• One new teacher, whose training is consistent with the vision advocated in the 
frameworks, explained that it is a comforting feeling to know that there is unity 
in the state on what good math and science look like. For her, the frameworks 
add validity to what she has learned. 

• Another teacher said that the frameworks help her plan instruction and help her 
avoid staying too long on one topic with which she feels comfortable. She also 
believes the frameworks and district curriculum foster communication among 
teachers at all schools and help unify the district. 

• A high school teacher believes the frameworks help her make modifications in 
her courses and introduce more skills-based instruction. 
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Framework use was not restricted to isolated groups of teachers on committees. As 
the following description suggests, whole-school change was well under way in the 
district. 



Exhibit 14 

AN EXEMPLARY GEMTOWN SCHOOL 



Bradley Elementary 

Throughout the recently renovated building, groups of students worked with peer 
tutors in wide-open spaces between classrooms. Student projects decorated the 
hallways and classroom walls. 

Last year, teachers were given a release day to examine the curriculum 
frameworks, and a lot of time was spent after school to discuss them in study groups. 
This year, the principal asked teachers to take the science curriculum framework and 
develop a notebook with ideas for instruction and assessment. Using the state 
curriculum framework and the district curriculum framework guides, teams of teachers 
created a school curriculum guide with blocks of study for each grade level. For 
example, in 3rd grade there were blocks covering space, electricity, etc. Second- 
grade blocks included sound and changes in prehistoric life. The notebook defined 
what strands need to be taught within each block and included sample assessments 
with problems that students need to solve and rubrics for scoring. The principal also 
asked all teachers to keep portfolios to pass on to the students’ next teacher. 

We were told that teachers start out each day with a problem. Most teachers 
reserved the afternoon block of 12:45-2:30 for science (rationale: students’ 
enthusiasm for science makes the afternoon hours ideal). Aside from this block, 
many other daily activities were linked to science. We saw evidence of science 
themes and integration on the walls of every classroom. In one classroom, stories 
about dinosaurs hung above collections of rocks, stones, and books about prehistoric 
life. Another teacher had her class reading a book about Thomas Edison and wove 
that into lessons on temperature, state geography, and social studies (which states 
are cold and hot). Another teacher organized an activity around their lOlst-day-of- 
school celebration: each cooperative group was given 101 bones to count out and 
piece together into a skeleton. This led to lessons on dinosaurs, plant/meat-eaters, 
and other related topics. The classroom and school hallways were lined with these 
skeletons. Our brief tour also uncovered a lot of cooperative learning and hands-on 
activities. Students sat around tables talking about and working together on projects. 
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Exhibit 14 (Concluded) 



Teachers participated in frequent professional development activities and were 
very enthusiastic. One teacher shared with us her transformative experience with the 
district’s museum sabbatical program. Wanting more content, she signed up and 
found that it sparked a new interest in science. The program made her feel what it 
was like to be a learner again, and she has been able to transfer that excitement and 
learning to her students. The program also provided her with new resources (e.g., 
kits, guides). She spoke about the power of science to allow students to try new 
things and help students struggling in other disciplines to gain confidence. All 
teachers were enthusiastic about the school. Another exuberant teacher bragged 
about her students’ growth and the changes she had undergone in terms of 
understanding that it was okay to tell students that she did not know ail of the 
answers. Without training in college in math and science, this teacher learned new 
skills and how not to be afraid of science. 

It is worth noting that this was considered to be the “neediest” school in the 
district, with 40 percent of students receiving free and reduced-price lunch. Bradley 
was outperforming other schools in the district. The principal boasted of “off the 
charts” scores on the open-ended portions of the Massachusetts Educational 
Assessment Program, as well as few discipline problems. 

The long-term reform efforts in the district appeared to result in higher student 
achievement (see Exhibit 15). Results from the 1994 and 1996 Massachusetts 
Educational Assessment Program (MEAP) showed that Gemtown 4th-graders made 70- 
point gains in mathematics and 90-point gains in science. Eighth-grade scores jumped 80 
points in both mathematics and science, and lOth-grade scores increased 80 points in 
mathematics and 50 points in science. (According to the state, real educational changes 
are detected when scaled scores rise or fall at least 50 points.) 

Exhibit 15 

GEMTOWN AND MASSACHUSETTS MEAP RESULTS 
(Scaled Scores) 





Grade 4 


Grade 8 


Grade 10 


Gemtown 


State 

Avg. 


Gemtown 


State 

Avg. 


Gemtown 


State 

Avg. 


1994 


1996 


1996 


1994 


1996 


1996 


1994 


1996 


1996 


Math 


1310 


1380 


1330 


1260 


1340 


1330 


1240 


1320 


1310 


Science 


1330 


1420 


1360 


1260 


1340 


1330 


1250 


1300 


1310 



Note: Scores provided by the Gemtown Public School System. MEAP scaled scores occur in a 
range between 1000 and 1600. 
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Cutler City, New York: Frameworks on the Context of Whole-School Change 

The Cutler City public schools enroll about 23,000 students in 27 elementary 
schools, 4 middle schools, and 4 high schools. The student population is growing 
rapidly, adding about 1,000 students per year. The district employs about 1,700 teachers. 

Through the evaluation of the New York SSI, the Pew Network for Standards- 
Based Reform, and the Eisenhower State Curriculum Frameworks Project, SRI 
researchers have closely followed the Cutler City School System since 1993. Cutler City 
schools became involved in standards-based reform with an early 1990s initiative called 
Education 2000. Education 2000 resulted in the drafting of a set of general standards and 
the development of benchmarks for those standards. Most district officials readily admit 
that the standards were too process oriented and too vague. Since then, the district has 
begun to focus on the New Standards along with the New York State Learning Standards 
(the mathematics, science, and technology section developed through the State 
Curriculum Frameworks Project). 

Because of the district’s involvement with the New York Statewide Systemic 
Initiative, district officials and some of the schools were involved in the development of 
the New York State Learning Standards. Cutler City has also committed to using the 
New Standards as the core of a professional development initiative sponsored by a large 
foundation. The district will administer the New Standards Reference Exam to students 
fi'om those schools that participate in the initiative. District officials are confident that 
the New Standards are equivalent to the state standards. Some differences are apparent, 
however. For example, unlike the New Standards, the state emphasizes the integration of 
mathematics, science, and technology. 

Although most schools in the districts are just beginning to use standards 
documents, several schools have been actively involved with standards-based reform for 
the past 3 years because of their involvement with the state’s SSI. One of those schools 
has made dramatic improvements in a relatively short time (Exhibit 16). 
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Exhibit 16 

A SCHOOL AHEAD OF iTS DiSTRiCT 



Smithson Eiementary Schooi 

Smithson Elementary School is a small school (about 600 students) that serves 
an ethnically diverse student body. The school stands out as having made 
remarkable progress toward changing the school culture and teacher practice in the 
vast majority of classrooms. 

As an elementary school with a preexisting disposition for reform and a focus on 
science, it began its participation in the standards-based reform with several 
advantages not found in most of the other schools in the district. One of the leaders 
of the reform effort was the union representative, a key ingredient for change with a 
strongly unionized teaching staff. Much of the reform effort was devoted to allotting 
time for teachers to read and reflect on research and new curriculum. Study groups 
became a regular feature of life in the school. Teachers were given opportunities to 
learn and discuss together in a supportive and safe environment. Key decisions 
concerning professional development, curriculum, and new initiatives were 
increasingly made by a steering committee of teachers. Recently, teachers have also 
instituted a Wednesday morning breakfast club where research and curriculum are 
discussed. In preparation for the New Standards Reference Exam and in anticipation 
of the new state assessments, Smithson Elementary School teachers have begun 
developing sample performance tasks for use in their classrooms. 

On the basis of countless interviews and observations over the past 4 years, it 
was clear that teachers became more confident about their practice, more protective 
of the school community, and, at the same time, better able to identify what they did 
not know. Leadership from the principal was important, but the school handled a 
recent change in principals with ease. The school has built a community of learners, 
fully aware of how much work they have ahead of them. 

Student achievement data seem to support the qualitative evidence of this 
school’s accomplishments. In the spring of 1994, only 75 percent of 3rd-grade 
students at the school scored above the SRP (State Reference Point) on the state 
mathematics test. By the spring of 1996, the figure was 93 percent. Similarly, 6th- 
grade students improved from 82 percent scoring above the SRP on the state 
mathematics test to 92 percent reaching that level. In science, 4th-grade students 
correctly answered, on average, 34 out of 40 questions on the manipulative section of 
the state test, 3 more than other schools with similar demographics. 
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Using Frameworks: Three Districts Working on Reform 

In the next set of three cases, we examine some districts that were actively using the 
frameworks but were not as far along in the reform process as our best-case scenarios. 
Here we applied the same criteria as in the best-case scenario cases and placed these 
districts in the “working on reform” category if they failed to meet one or more of the 
criteria. 

Parkside School District, Maine: Standards-Focused Professional 

Development 

The state framework is having a significant impact on science education in Parkside 
district, located in a small town in Maine. The 1,700-student district includes one high 
school, four K-8 schools, and three one- or two-room schools located on islands just off 
the coast. The district employs about 150 teachers. 

When the state framework was published, Parkside was using its own locally 
developed science framework. After comparing the two documents, Parkside educators 
decided to build a new local framework using the state framework as a starting point. 

The local framework development and related curriculum activities have been facilitated 
by opportunities for substantial, ongoing professional development. The local curriculum 
work is also a professional grov^dh experience, apart from the formal training that has 
accompanied it. 

For the past year, Parkside teachers have been comparing what is being taught at 
each grade level with the state’s standards in all content areas. Every teacher in Parkside 
is required to serve on one of the content-area subcommittees. The science subcommittee 
consists of 28 members and includes teachers from all district schools. Supported by 
LEA and SSI funds (the Parkside district is one of seven SSI sites in the state), grade- 
level or grade-level-cluster teams met during the summer of 1996 to work on classroom 
curricula and assessments. At the K-8 level, this effort included aligning and developing 
new kit-supported science units. 

An example of this process comes from the work of three kindergarten teachers. 
They worked together to analyze the five science kits that had been developed several 
years before with local grants as part of the SSI inservice program. The teachers chose to 
work on the “water” and “critters” kits for their 3-day summer meeting (the other kits 
have been analyzed since then). Activities proceeded as follows: 
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Day 1 : The team matched the kits to the state content standards, noting activities 
that could work for assessment (performance indicators). They also matched 
community, equity, and professional development standards to each kit. 

Day 2: Teachers decided to add a content standard for the “water” kit — one that was 
not addressed in the state framework. The team researched the new standard using 
the national science standards, the California Benchmarks, and at least one other 
publication. They then developed an observation checklist to support teacher 
observation of science inquiry by students. 

Day 3: Teachers finalized the mapping so that each kit was associated with the state 
standards and assessments, and wrote some recommendations for use of the kits by 
other kindergarten teachers. 

During this work, the team found that a number of performance indicators, 
representing eight content standards in the framework, were not met through the kits. 

The team then wrote a report for the district detailing the situation and recommending 
that these be addressed through other classroom activities. 

The one district high school has also been involved in related curriculum revisions. 
The high school, which enrolls 540 students, offered applied science courses, college 
preparatory courses, and honors courses. A new 9th-grade science course, called 
Foundations of Science, was developed by two local teachers on an “every science, every 
year” model. The course included topics in biology, chemistry, and physics and was 
often interdisciplinary. The science department chair reported that science course-taking 
patterns have changed at the school in recent years. Many more students were taking 
college preparatory biology rather than general biology, and enrollments in a third and 
fourth year of science were up to the extent that the school has just added a new 
laboratory. 

At the K-8 level, a study group met monthly on science curriculum articulation 
across grade levels. The outcome of this process so far was that the science curricula at 
the elementary and middle school levels were parallel. Ongoing work includes checking 
for areas of overlap or omission across grade levels. 

The curriculum alignment, revision, and articulation activities described above are 
important opportunities for professional growth. Yet an additional component that 
accounts for the success of these activities is the formal professional development that 
most of the participating teachers have received. Indeed, when Parkside teachers began 
the activity of aligning the curriculum with the framework, no one was experienced in 
how to perform the task. Teachers’ abilities to develop science curricula have been 
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enhanced by their participation in statewide working conferences and 2- to 3-week 
Academies sponsored by the SSI (Parkside is one of seven SSI Beacon sites in the state). 
A 2-day inservice was held in November 1996 for the science subcommittee. Among the 
goals of the activity were to develop a progress timeline; to report on the previous 
summer’s work of comparing existing units with the framework; to make the connection 
between performance indicators, assessment, and teaching practice; to better align scope, 
sequence, and coordination across grades; to link the Foundations of Science syllabus to 
the framework; and to revise the 5-year plan. 

District professional development will continue to focus on developing an 
articulated science curriculum by helping teachers identify curriculum holes and 
duplications for their immediate grade clusters. In addition, understanding that the 
framework has pushed them toward very different kinds of classroom assessment, 
individual teachers are pursuing professional development in that area. In the spring of 
1997, several district teachers were taking an assessment course offered by an SSI 
facilitator for University of Maine graduate credit. 

Most Parkside teachers appeared to be happy with the attention to curriculum and 
the changes it has brought about. Middle school teachers said that the matching/mapping 
activity is the first time the curriculum has gotten serious attention in a long time and that 
Parkside’ s standards are now stronger than the state’s. They also indicated that there is 
now more teamwork, more planning time, more professional development, and more 
contact with colleagues at other schools. According to one respondent, interteacher 
communication is much greater now than 5 years ago. Teachers are less competitive and 
more collegial. One teacher mentioned proudly that several Parkside teachers had been 
asked to take leadership roles in statewide conferences, like the intensive, well- 
established, state Problem Solving in Science and Math (PRISM) conferences. This 
counts as another professional development opportunity for the teachers involved. 

As in the best-case scenarios, Parkside teachers were deeply involved in a variety of 
professional development activities. The difference between the first two cases and 
Parkside appeared to be a matter of time. But given adequate time to build capacity, the 
district seemed poised to make significant improvements. 

Farmville Public School District: School-Level Use of Nebraska’s 

Mathematics and Science Framework 

Farmville, Nebraska, is a small community of about 1,000 people. The entire K-12 
population of about 300 is housed in one building, along with the district offices. About 
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half of the students receive free or reduced-price lunches. Nearby World War Il-era 
military barracks have recently been converted into low-income housing. Many of the 
low-income residents are migrant workers who stay in the community for varying lengths 
of time. This situation has led to greater demands on the K-12 school and greater 
mobility for the student population. Enrollment per grade level averages between 20 and 
25. Typically, a graduating class includes about 5 students who attend a 4-year college 
and 1 0 students who attend the nearby community college. 

The district received copies of the Nebraska Mathematics and Science Framework 
about 2 years ago. The state science coordinator provided the district with a workshop on 
the frameworks. A few teachers from the school were involved with the state’s SSI and 
first used the frameworks during SSI-sponsored workshops. The region’s Education 
Service Unit also provided some teachers with a workshop that focused on the 
frameworks. More recently, the state science coordinator visited the district at the 
district’s request to help implement the frameworks. 

The district’s curriculum revision committees, which included teachers, 
administrators, and members of the community, were the primary users of the 
frameworks. The math and science framework has been used as a reference tool by the 
committees that have been revising curriculum in math and science. Teachers cited the 
two large charts (one each for math and science) that organize “conceptual threads” and 
“topic strands” by grade-level cluster and the suggested activities as being “very helpful” 
to them. The principal of grades 4 through 12 has also used the charts and the 
frameworks to justify the need for changes in classroom instruction when he confers with 
teachers. 

The 4 through 12 principal and teachers were pleased with their progress. The 
principal mentioned seeing more hands-on activities in classrooms and seeing “students 
excited about learning.” Teachers reported teaching more hands-on lessons. Several also 
said that they have used the activities included with the framework. (A 1995 Addendum 
to the 1994 framework document is made up of sample lessons in math and science for 
different school levels.) Teachers also referred to new ways of teaching and learning. 

Two middle-level teachers were pleased that they did not have to be the only “teachers,” 
since their students were also learning from each other with the new activities. Students, 
according to these teachers, liked being more in charge of their own learning and the 
discovery process. Students also knew that “it’s all right to fail if you learn.” During our 
observation of a lesson on body systems, one student suggested that this was just like the 
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checks and balances system in government (a recent history topic for the class). Another 
student commented that they did science during math class that morning. A discussion 
occurred about relationships and how things work together. High school teachers at 
Farmville School are not as involved with the frameworks as their middle-level 
colleagues, but they did mention using more hands-on and cooperative learning strategies 
in mathematics and science. 

The impact of the frameworks in this district was most evident in the curriculum 
revisions and in the teachers who have been active participants in workshops and other 
professional development opportunities related to the frameworks. The 4 through 12 
principal believed that the framework was an excellent guide for their curriculum and was 
pleased with the integration and alignment of the curriculum that resulted from the 
revision process. Frameworks, in his words, “opened up information teachers didn’t have 
at their fingertips.” The high school math teacher credited the framework with getting 
people to take workshops that they might not have attended otherwise. He also 
mentioned the ties between math and science that are now in place in the curriculum. 
When teachers look for textbooks, according to several respondents, they compare the 
content with the frameworks. 

In this small town, the district’s use of frameworks was more individualized and 
less planned than Parkside’s. Although district officials argued that their revised 
curriculum was evidence of effective framework use, classroom-level implementation 
was uneven. 

Red Ridge, Arkansas: Frameworks as a Distraction 

The Red Ridge School District serves about 2,800 students. The district has five 
elementary schools (K-4), one middle school (grades 5-6), and one high school or 
“secondary complex” (grades 7-12). The Red Ridge area has traditionally been a 
manufacturing center with low -wage, low-skill jobs. More recently, new industry that 
requires more skilled workers has moved to the region. District officials saw the need to 
equip students with better mathematics and science skills so that they could graduate 
qualified for college or the better-paying, high-skill jobs. 

Red Ridge had a reputation as a progressive and innovative district with a dynamic 
superintendent. Much of its reputation stemmed from the host of brand-name reforms 
under way in the district. The district was involved with New Standards, Schools for the 
2 1 st Century, the Coalition of Essential Schools, and one of the New American Schools 
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initiatives. Individual schools also focused on a variety of curricular and other reform 
ideas. District officials worried about the overload of reform initiatives, but found it easy 
to get new grants once the first one was won. The following case describes one of the 
more reform-oriented schools in the district. 



Exhibit 17 

REFORM WiTHOUT FRAMEWORKS 



Meadows Eiementary Schooi 

Meadows Elementary School is located next to Meadows Middle School about 
3 miles from downtown Red Ridge. It consists of a small central school building and 
portable classrooms, a gymnasium, and a new cafetorium. The school sits on 30 
acres of land, surrounded by small farms. It serves about 360 students in grades 
K-4 with 25 teachers, 2 aides, a guidance counselor, and a librarian. 

The hallways of the school were decorated with various environmental scenes: 
the desert, the mountains, Arkansas, and the ocean as part of the schoolwide 
“Regions of the Earth” theme. The displays included items found in each region, a 
large mural, and, in some cases, tape-recorded sound effects. Most of the 
classrooms were equally colorful, with lots of displays of children’s work and evidence 
of a theme-based approach. The schoolwide themes included: Our Heritage, 
Arkansas, Pioneers, United States Geography, Science Fair, Regions of the Earth, 
and the History of Our County. The school had a welcoming and child-centered feel 
to it. 

The principal seemed to be a strong and hard-working leader. She has devoted 
much of her time and energy to acquiring special funding for the professional 
development of her teachers. She drove van loads of teachers to Kentucky to 
observe schools. She found funds to fly teachers to national conventions and to 
exemplary schools as far away as Vancouver, BC. Another group of teachers were 
sent to Cincinnati to observe classes with multi-age grouping. 

The Meadows School brochure listed an incredible array of programs, including 
12 special instructional programs [accelerated reading. Box It Bag It Math (an 
extension of Math Their Way), developmental kindergarten. Sing Spell Read Write, 
Shurley Method, Success in Reading, K-4 Crusades, etc.]; 16 service programs 
(mentoring programs, drug education, summer enhancement, parent volunteers, 
positive action program, Reading is Fundamental, Wee Deliver, etc.); 8 learning 
environment programs (cooperative learning, whole language, thematic units, 
literature-based reading, hands-on manipulatives, peer tutoring, teacher teams, etc.); 
and an outdoor environmental laboratory. Project MAST pilot school, and Partners in 
Education (business partnerships). 
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Exhibit 17 (Conciuded) 



On top of all this reform activity, the state’s new curriculum frameworks arrived in 
the district. Two of the school’s teachers volunteered to serve on the district 
committee to revise the curriculum on the basis of the framework. Unfortunately, the 
experience failed to bring any coherence to a clutter of projects and reform initiatives, 
and they expressed frustration with the committee’s work. 



In Red Ridge, the Arkansas mathematics and science curriculum frameworks got 
lost among the many reform initiatives. Like other districts in Arkansas, the state 
department of education asked Red Ridge to revise its district curriculum in light of the 
new state frameworks. Red Ridge organized a committee of teachers representing all 
grade levels. The committee received samples of revised district curriculum guides from 
districts that were supposed to have piloted the new frameworks. However, the sample 
guides were essentially long lists of textbook pages associated with the various strands of 
the framework. The committee broke into elementary, middle, and high school 
subcommittees, who were charged with revising the district curriculum guide for their 
grade levels. The elementary school group completed an extensive document; the middle 
and high school groups took the task less seriously, completing only one page each. 
District officials reported that the eventual result was that the framework activity was just 
a distraction. They also pointed to districts that revised their district curriculum by 
photocopying one from a neighboring district. 

During this period of active reform in the district, students’ test scores on the 
Stanford-8 Achievement Test dropped, while less innovative neighboring districts’ scores 
were going up. District officials argued that the neighboring districts were teaching to the 
test. In addition, they decided that teachers were spending too much time out of the 
classroom in professional development. As one district administrator declared, “It 
concerns me very much that a generation of kids are sitting in class with a sub rather than 
a certified teacher.” Most teachers argued that the tests simply did not measure what they 
were teaching in their classrooms. The district is trying to continue its reform orientation, 
but public pressure to boost the test scores is strong. District officials hope that their 
students will fare better on the new state assessments. 
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Cross-Cutting Themes 



Translating State Frameworks into Local Documents 

All five cases, Gemtown, Cutler City, Parkside, Farmville, and Red Ridge, reflected 
a pattern of activity common all across the country. It appears that every level of the 
system is involved in creating frameworks or guides based on standards. On the face of 
it, this emphasis on document production seems redundant. Yet we found some genuine 
benefits to these activities, as well as a few challenges. As we reported in previous 
reports, those who participated in the creation of the state curriculum frameworks seemed 
to derive the most benefit. Given that fact, a certain degree of redundancy may have been 
a necessary price for wider participation in standards-based reform. Of course, as the Red 
Ridge case illustrated, local standards development can be poorly done. Still, our district 
cases support the view that professional practice can improve when teachers and others 
are involved in production of local standards. 

Experience with Reform 

The examples of Gemtown and Cutler City, our best-case scenarios, provide further 
insight into the effective use of frameworks. Both districts and the highlighted schools 
engaged the standards documents from a foundation of previous reform activity. The 
Gemtown district was particularly active in a districtwide strategy for change. At the 
same time, the Smithson Elementary School was well on its way to schoolwide reform 
when the teachers began to use the standards documents. In both cases, the teachers were 
not confronted with unfamiliar concepts when they began studying the documents. For 
example, at Smithson we observed a workshop designed to introduce standards and heard 
a collective sigh of relief as teachers began reading the documents. 

Whole-School Change, Collegiality, and Professional Development 

In addition, our best-case scenarios point to the importance of whole-school change 
as a supportive condition for effective use of the frameworks. In these cases, teachers 
were able to experiment with the ideas suggested by the framework in a collegial and 
professional atmosphere. Teachers served as resources to each other and worked together 
on projects. In addition, the school culture supported the idea of learning as a collective 
endeavor. 

Professional development activities designed to help teachers use the state 
frameworks in Gemtown, Cutler City, Parkside, Farmville, and Red Ridge varied widely 
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in duration, intensity, and coverage. Of the three districts just beginning to use standards, 
Parkside teachers obviously had the most intensive and extensive opportunities to use the 
frameworks. Whereas Parkside required every teacher in the district to serve on one of 
the content-area subcommittees, Farmville and Red Ridge teachers’ participation in 
framework-related professional development was completely voluntary. One result was 
that Farmville’s high school teachers were not very involved with the frameworks or the 
district’s efforts to revise curriculum. In Red Ridge, a select group used the 
frameworks — only those teachers on the revision committee and those teachers 
introduced to the frameworks through professional development opportunities outside the 
district. 

In our best-case scenarios, continuing professional development opportunities were 
abundant and increasingly focused on standards. Linking professional development to 
framework and standards documents helped bring coherence and direction to teachers’ 
professional growth. Although teachers could have used more time during the regular 
school day to plan, study, and reflect, professional development was a routine part of 
teachers’ work. In Gemtown, the district built incentives for continuing professional 
development into the teacher contract. At Smithson School, the teachers played a central 
role in the planning of their own professional development. 

As recent research on instructional reform concludes, the reform messages 
transmitted are not necessarily the messages that teachers receive or act on in practice 
(Spillane & Jennings, 1997). Even in the more comprehensive professional development 
strategy under way at Parkside, the teachers had not had enough time to use and reflect on 
the standards to consistently incorporate the ideas into their practice. Clearly, they were 
not as far along as the teachers in our best-case scenario districts. Even in the Gemtown 
district and the Smithson School, the teachers recognized that deep changes in their 
teaching required continuing professional growth. 

Although our cases underscore the importance of including intensive, ongoing 
professional development as part of the cost of operating a school, none of the sample 
districts were particularly well funded. Professional development remains vulnerable to 
budget cuts. Beyond limited resources, the districts were only beginning to explore ways 
to build professional development into the structure and organization of the school day. 

In Cutler City, teachers in a middle school not far from Smithson recently devised a 
school schedule that allowed core teachers 2 hours of common planning time each day. 
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Although the school principal supported the plan, the district balked, not wanting to set a 
precedent. 

Adaptation vs. Adoption 

In each of the five cases, districts adapted the standards and framework documents 
rather than adopted them. Local districts will inevitably adapt guidance from the top of 
the system to suit local conditions, but in our best-case scenarios the schools and the 
teachers had the capacity to add value to the guidance during the adaptation process. In 
the Gemtown district and the Smithson School, long-term experience with standards- 
based reform, extensive professional development opportunities, and an evolving 
professional culture helped create high-capacity organizations able to use the framework 
well. Thus, these cases suggest that reformers at the top of the system are always 
dependent to some degree on reformers at the bottom of the system — suggesting that 
capacity building is a key to successful implementation. 

Content vs. Pedagogy 

Although there are clear benefits to involving large numbers of teachers and 
administrators in the production of standards-based documents, there remain some 
unresolved issues. First, our district cases suggest that local involvement with state 
frameworks emphasizes content over pedagogy. Much of the work of the various 
curriculum committees, study groups, and workshops involved translating the state 
frameworks into grade-by-grade local curriculum guides with few references to teaching 
strategies. Only in schools like those highlighted in the first set of cases, Bradley and 
Smithson, did we note intensive teacher attention to both content and pedagogical issues. 
In both cases, the school climate supported teacher conversations about such pedagogical 
issues as teacher attitudes and behaviors, teaching style, classroom management in an 
activity-oriented classroom, developing and using performance assessments for 
instructional purposes, and attending to cultural differences among students. 

The imbalance between the need for changes in both what is taught and how it is 
taught stems, in part, from states’ reluctance to offer guidance on pedagogical issues. 
Even in states that produced framework documents with extensive pedagogical guidance, 
it was not always easy to get districts to pay attention. For example, Massachusetts’ 
highly regarded state curriculum framework includes a first chapter (separately bound) 
that discusses pedagogical issues that are common to all of the disciplines, and six 
chapters devoted to English/language arts, mathematics, science and technology, history 
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and social studies (currently under review), the arts, and health. Our informants reported 
that in high schools and middle schools the discipline chapters were sometimes handed 
out to the appropriate discipline teachers, but no one received the common chapter. 

Assessment and Accountability 

Each of the cases suggests continuing challenges for standards-based reform for 
even the most positive cases. Although our best-case scenarios described schools that 
had improving test scores, teachers were still struggling with the sometimes conflicting 
purposes of assessment. On the one hand, teachers were trying to devise their own 
assessments for instructional purposes and found the task challenging and time 
consuming. On the other hand, the existing state assessment systems in both New York 
and Massachusetts were not yet well aligned with the state frameworks. Both states are 
developing new assessments, but teachers felt pressure to devote more attention to tests 
with accoimtability purposes than to tests with instructional purposes. The technical 
challenges involved in large-scale assessment make it imcertain whether the new 
assessment systems will be able to resolve this dilemma for teachers. 

A Remaining Challenge 

As the five cases suggest, standards-based reform has captured the attention of 
many professional educators who are trying to improve mathematics and science 
education. However, as we indicated earlier and saw again in the five cases, standards- 
based reform has been primarily a professional, not a public, activity. This lack of public 
involvement with standards is a problem because, regardless of their effectiveness, reform 
leaders — from the superintendents to the principals — ^were vulnerable to shifts in public 
opinion and the political composition of the local school boards. Without stronger public 
support, even the most promising reform efforts could be scuttled. 

Although the public seems to approve of the concept of standards for student 
performance, public understanding of the standards has been superficial, at best. Even in 
our best-case scenarios, parents that we interviewed expressed support for the work of the 
schools but little understanding of the standards that were guiding that work. In both of 
our best-case scenario districts, efforts were under way to familiarize parents and the 
wider public with the standards, but these efforts tended to be one-way information 
sessions. Just as teachers and administrators in the 5 cases and the 17 districts we visited 
enhanced their understanding of standards and frameworks by using them to create local 
curriculum documents, parents and the public are also likely to need more than brief 
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introductions. Particularly at the school and district levels of the system, there are 
significant opportunities to engage parents and the many publics, in partnership with 
teachers and administrators, in the hard work of defining what children should know and 
be able to do. 

As one observer noted several years ago: “Whether standard setting at any level of 
government can foster improvements in the education of the nation’s children will depend 
on the quality of debate standard setting engenders. More specifically, it will depend on 
whether such debate can help to educate the public about education and, beyond that, 
assist in mobilizing greater interest, appreciation, and concrete support” (Lagemann, 
1995). This is a different measure of success than is typically associated with standards- 
based reform. Like the participants in the fi'amework projects, reformers defined success 
as increased school effectiveness and higher student performance. The implicit 
assumption is that if the schools are improved, public support will take care of itself. But, 
as another commentator pointed out recently, the public schools are at risk because they 
have lost their legitimacy, not just their effectiveness. “Institutions face a loss of 
legitimacy when those who created them no longer believe that the institutions are their 
agents, acting on their behalf . . . Like a crack in the foundation of the public school 
system, the lack of legitimacy is a structural defect that undermines all the good work to 
make the schools more effective” (Mathews, 1997). 

The standards-setting efforts we found in our sample of local districts have engaged 
some educators focused on school effectiveness, but not on issues of legitimacy. 

Although professional educators are making well-motivated efforts to boost expectations 
and improve student performance, the public debate is about charter schools, choice, and 
vouchers. The public is increasingly convinced that nothing short of a major overhaul of 
the system will do the job. One recent survey reported that 56 percent of the public 
believed that the schools had completely failed or that they should be overhauled (Belden, 
Breglio, Keman-Schloss, & Plattner, 1997). 

For standards-based reform to succeed, reformers must simultaneously address 
issues of public school effectiveness and legitimacy. To that end, standards setting needs 
to be more than a professional activity, and it must involve the parents and the various 
publics in addressing such difficult issues as: “How do we ensure that all children meet 
the standards?” “What is the proper balance between teaching the basics and promoting 
higher-order thinking?” “What gets left out of the curriculum if we replace breadth with 
depth?” 
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Engaging the public in standards-based reform efforts will not be easy. Educators 
are much better at walling off the public from “professional” concerns than engaging the 
public. Only by broadening participation and broadening the debate can standards-based 
reform begin to address both effectiveness and legitimacy. As Robert Bellah and his 
associates (1991) pointed out, the pressing question for American government “is not just 
what government should do but how it can do it in a way that strengthens the initiative 
and participation of citizens, both as individuals and within their commimities and 
associations, rather than reducing them to the status of clients.” Meaningful public 
engagement is a remaining challenge for standards-based reform if it is to last and if it is 
to matter. 
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Conclusion: Standards-Based Reform at the District Level 

We began this chapter with the caution that it was still too early to see widespread 
changes in policy and practice at the district level. The districts that we have described in 
this chapter, along with the others that we visited, reflect localities that have been using 
their states’ frameworks. State officials are confident that other districts are beginning to 
be actively engaged in standards. They insist that the states’ curriculum frameworks will 
become increasingly important as new assessment systems are introduced or as the 
professional development in the states becomes more consistently focused on standards. 
Other recent studies support this view. For example, the 1997 CPRE report Persistence 
and Change: Standards-based Reform in Nine States found that “half of the districts 
located in states with standards in place reported that the standards initiatives had 
influenced their own instructional guidance efforts” (Massell, Kirst, & Hoppe, 1997). 

The majority of the district cases paint a fairly positive picture of what happens to 
framework documents once they reach the school districts. At the same time, the cases 
underscore the amount of time and resources needed to use frameworks well. At Bradley 
and Smithson, teachers had a solid foundation of reform experience, good leadership, and 
adequate resources that allowed them to use the framework and standards documents in 
meaningful ways. In both schools, this foundation was built over time. 

Even with a solid foimdation, schools and districts must confront a large number of 
issues if standards-based reform is to mature and result in systemwide improvements. 

We pointed to the challenges teachers faced in trying to balance the accountability and 
instructional purposes of assessment. To the extent that new assessment systems 
encourage teachers to prepare students for the new tests in old ways, the new assessments 
may only reinforce a culture of testing and tracking. We also argued that meaningful 
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parent and public engagement with the standards is a largely unaddressed challenge for 
standards-based reform. As the cases make clear, effective local standards-based reform 
requires significant amounts of time and resources. The effectiveness of standards-based 
reform will depend on how well a public debate about what students should know and be 
able to do translates into a public commitment to provide what it takes to allow all 
students to meet those expectations. 

In the concluding chapter, we will review the major findings of the study and 
discuss the implications of those findings for the federal strategy. 
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VIII. CONCLUSIONS 



This final report has used findings fi-om the evaluation of the 1 6 state curriculum 
fi-amework projects to better understand the implementation of standards-based reform at 
the state and district levels. In this concluding chapter, we summarize the findings of the 
report and discuss implications of the findings for the federal efforts to promote high 
standards for all students. 

Summary of Findings 

As is the case in the vast majority of states, all 16 states that received curriculum 
fi-amework grants were actively engaged in standards-based reform. Even in those states 
where the fi-amework projects ran into trouble, state officials and policy-makers were 
busy reviewing and revising policies and programs under the banner of standards. The 
robustness of standards-based reform helped make the projects’ curriculum fiiameworks 
useful resources to the states. Fifteen of the 16 projects completed curriculum 
fi-ameworks in mathematics and/or science. 

The projects devoted less time and resources to the project’s other components: 
model guidelines for teacher education and certification, criteria for teacher 
recertification, and model professional development. As a result, the projects were 
unable to meet the full expectations of the original solicitation. The projects’ uneven 
progress on these other products was caused by the unrealistic assumptions of the original 
solicitation and aggravated by the projects’ emphasis on fi-amework development. 

The projects’ fi-amework development process typically involved large numbers of 
educators in the writing, review, and revision of documents. Framework development 
required educational expertise, but the process also encountered political issues. In 
adapting to each state’s political context, the fi-ameworks avoided violation of local- 
control sensibilities and other politically contentious issues. In the end, fi-amework 
development was a professional, rather than a public, activity. 

A review of a sample of fi-amework documents by outside experts found that 
although the fi-ameworks acknowledge the influence of the national standards, many 
documents omit major content categories or rewrite particular standards in such general 
language that the original meaning is lost. The reviewers did cite some exemplary 
features of specific documents but were critical of many of the sample activities, the use 
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of jargon, and the lack of guidance on equity issues. Our subsequent review of local 
standards documents found great variation amonig the local attempts to align their 
curriculum guides and local frameworks with the state frameworks. 

Our examination of the frameworks’ role in state policy revealed that states were 
actively engaged in two key policy areas, assessnient and teacher licensure, with or 
without the help of the projects’ frameworks. Fourteen of the 16 states were actively 
developing new assessment systems. However, only 10 of the 16 states used the projects’ 
frameworks to develop these systems. Similarly, 9 of the 16 states had implemented or 
were developing new teacher licensure policies; 6 of these states used the projects’ 
frameworks to develop such policies. 

The 16 project states employed a variety of strategies to use the frameworks to help 
improve teacher practice. All states used framework-based professional development as a 
primary strategy. Some projects recognized the need for additional support materials and 
the value of linking the frameworks to other resources and other initiatives. States’ other 
strategies included the use of technology and the creation of pilot sites. 

Our examination of the use of frameworks in a sample of districts and schools 
found that production of local standards can be a meaningful professional development 
activity. Effective use of frameworks was enhanced by previous experience with reform, 
whole-school change that helped create a collegial and professional school culture, and 
extensive and intensive professional development opportunities focused on standards. 

The cases also underscored the fact that districts and schools adapted rather than 
adopted standards and framework documents. The standards can be weakened during the 
adaptation process, but in our best-case scenarios the schools and the teachers had the 
capacity to add value to standards. 

The district cases also revealed that frameworks were used to address content issues 
far more than pedagogical issues. In addition, teachers in the districts were struggling to 
balance the accountability and instructional purposes of assessment. Finally, the district 
cases suggested that a remaining challenge for standards-based reform is better public 
understanding of standards and subsequent support for the resources necessary for all 
students to meet the standards. 
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Implications for Federal Efforts to Promote Standards-Based Reform 

Fourteen years after A Nation at Risk and 8 years after President Bush and the 
nation’s governors settled on the National Educational Goals, the so-called standards- 
based reform movement seems to have remarkable staying power. The State Curriculum 
Frameworks Projects have provided support for that movement in 16 states. The majority 
of the project states are using the fi’ameworks to revise key education policies, a clear 
contribution of the federal support. The impact of the projects is likely to continue as the 
fi’ameworks are used by local districts and other ongoing initiatives. 

Frameworks can serve as general policy guidance and can be useful to local districts 
in their reform efforts. Although federal funds lent support to standards-based reform in 
the states, each state’s political and educational context dictated what role the projects 
would play in the reform effort. In addition, the projects’ efforts on the other products 
revealed that teacher education, certification, recertification, and professional 
development policies involve a variety of interested parties and contentious political 
issues that are unlikely to be guided by theoretical documents produced in isolation fi’om 
those parties. The projects’ uneven efforts to develop model professional development 
programs, model guidelines for teacher education and certification, and criteria for 
teacher recertification also suggest a need for future ED grants to be based on a more 
realistic set of assumptions and a more reasonable scope of work. 

Much more work is needed before the curriculum fi’ameworks will be well used in a 
majority of districts and schools. As our case study districts suggest, capacity building is 
a key to successful implementation. Districts and individual schools need more time and 
resources to translate the state fi’ameworks into local curricular guidance, a process that 
makes the standards more meaningful. 

Given the need for capacity building at the local level, the inevitability of local 
adaptation, and the difficult issues facing teachers and administrators as they try to use 
standards to guide their practice, federal support for standards-based reform is most 
needed at the local level. Clearly, federal support through Goals 2000, the Eisenhower 
Professional Development Program, and other federal programs is already playing a role, 
but far more time and resources are needed than are currently available in most schools. 

Federal support for further curriculum fi’amework development at the state level is 
unlikely, and probably unnecessary, but our research makes it clear that teachers and 
administrators can benefit fi’om using the state curriculum fi’ameworks to revise local 
curriculum documents and reflect on their practice. Unfortunately, most schools lack the 



resources to provide adequate time for teachers to undertake such work. Making the 
standards more meaningful and understanding the implications of the standards for 
teaching cannot be done quickly and cheaply, and building public support for standards 
caimot be done from Washington, DC. Federal support for local implementation of 
standards would build on the work of the State Curriculum Frameworks Projects and help 
expand the number of schools that are using standards to create fundamental changes in 
the way they work. 
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EVALUATION METHODS 



In June 1994, SRI produced the Study of the Dwight D. Eisenhower Mathematics 
and Science Education State Curriculum Frameworks Projects and Regional 
Consortiums Program: Study Design and Data Collection and Analysis Plan. That 
document presented a theoretical framework for the evaluation, the study’s key research 
questions, and a detailed description of evaluation methods. In addition, the document 
included our data collection instmments, including case study guides and tailored topic 
guides for semistructured interviews. In this appendix, we present the research questions 
and summarize the data collection activities. 

Research Questions 

The original research questions that guided the study are presented in Exhibit B-1. 

Exhibit B-1 

RESEARCH QUESTiONS FOR THE STATE CURRiCULUM 
FRAMEWORKS PROJECTS 



Organization and Development of the State Curriculum Frameworks (SCF) Projects 

• How well did the SCF projects complement other significant education reform efforts in the states? 

• What is the relationship between SCF projects and other systemic initiatives in the state? 

• What state characteristics promoted successful SCF project activities? 

• How did activities in states with Curriculum Framework grants differ from states without these federal 
grants? What difference did the federal support make? 

• How did the development of curriculum frameworks in mathematics and science differ from the 
development of frameworks in other disciplines in the states? 

• How did the distribution of resources vary across the SCF projects? 

• How did the process of developing curriculum frameworks vary across states? 

• Who was included in the development of the curriculum frameworks? Who was excluded? 

• How did the states balance the need for expertise in the development of curriculum frameworks and 
the need for broad participation in the implementation of the frameworks? 

• How successful were SCF projects at building consensus about mathematics and science education 
reform? How widespread was participation in the projects? 
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state Curriculum Frameworks Projects Products and Services 

• How did the states define a curriculum framework? How do the curriculum frameworks differ 
between the states? 

• How many states developed curriculum frameworks and completed their other activities? What were 
the major barriers? 

• Was there any organized resistance to the development of curriculum frameworks or other SCF 
project activities? What is the position of those in opposition? 

• Where did the ideas for the specific design of the curriculum frameworks and other related initiatives 
come from? Did the states use existing frameworks as a guide? 

• What is the extent and impact of assistance and collaboration between the SCF projects and the 
Regional Consortia? 

• Are the states’ assessments, teacher preparation and accreditation, textbook adoption policies, staff 
development, and technology policies aligned with their curriculum frameworks? 

Quality and Effectiveness of the State Curriculum Frameworks Projects 

• To what extent are the frameworks and framework-related activities consistent with emerging national 
standards in mathematics and science education? 

• Which SCF project activities contributed to improvements in mathematics and science education? 
Which were less successful? 

• What evidence exists to suggest that completed curriculum frameworks and other policy changes are 
being implemented? 

• How have the SCF projects promoted changes in state and local policies affecting mathematics and 
science education? 



Data Collection Activities 

Data collection activities over the course of the 4-year study (1993-1997) were 
timed to maximize our understanding of both the development and implementation of the 
projects’ products. During the 2nd, 3rd, and 4th years of the study, data collection 
activities included: 

• Review of State Curriculum Frameworks Projects documents, including original 
proposals, continuation proposals, draft and completed frameworks documents, 
and draft and completed documents from model guidelines for teacher education 
and certificatipn, criteria for recertification and model professional 
development, and available evaluation materials. 

• Review of state data from a variety of secondary sources (see complete list of 
sources in Appendix A). 

• Telephone interviews with project directors, state officials, SSI directors, 
Eisenhower state coordinators, and key participants. 
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During the 2nd and 3rd years of the study, we also worked with a group of outside 
educational experts to evaluate the quality of the framework documents. In addition, we 
conducted 2-day site visits to four states (Florida, Louisiana, Nebraska, and Oregon) 
during the 3rd year of the study. In the 4th year of the study, we conducted more 
intensive week-long site visits to 8 of the 16 states. Those site visits included in-depth 
interviews with state officials as well as teachers and district officials in a sample of two 
to three districts in each state. Exhibit B-2 summarizes the evaluation’s data collection 
activities. 



Exhibit B-2 

DATA COLLECTION ACTIVITIES 



State 


1996 Site 
Visit 


1997 Site 
Visit 


Document Review 
and Teiephone 
Interviews (1995-97) 


Expert 

Review 


Site Visits 
through Other 
Studies* 


Alaska 






X 


X 




Arizona 






X 






Arkansas 






X 


X 


SSI 


Delaware 




X 


X 


X 


SSI, Pew 


District of 
Columbia 






X 






Florida 


X 


X 


X 


X 


SSI 


Louisiana 


X 


X 


X 


X 


SSI 


Maine 




X 


X 


X 


SSI 


Massachusetts 




X 


X 


X 


SSI 


Michigan 




X 


X 


X 


SSI 


Nebraska 


X 


X 


X 


X 


SSI 


New Jersey 






X 


X 


SSI, AAAS 


New York 






X 




SSI, Pew, 
AAAS 


Oregon 


X 


X 


X 


X 


Pew 


Rhode Island 






X 


X 


SSI 


Wisconsin 






X 







*SSI = statewide Systemic Initiative (National Science Foundation) 

Pew = Pew Network for Standards-Based Reform (Pew Charitable Trust) 
AAAS = American Association for the Advancement of Science’s Project 2061 



Document and Secondary Source Review 

Document and secondary source review was an important part of the data collection 
activities. A complete list of documents and secondary sources was prepared by SRI and 
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included in an April 1997 report to ED entitled “Document Lists, Telephone Interview 
Records, and Selected Indicators.” Sources and documents related to the State 
Curriculum Frameworks Projects are included in Appendix A. 

We gathered successive drafts of framework documents and thereby imderstood 
how these frameworks were developed and revised in different states. We also collected 
state documents and other materials that influenced the frameworks (or from which the 
frameworks represented a significant departure). Typically, these were state curriculum 
guides or lists of goals and objectives for mathematics and science. Examples of 
additional documents included: 

• Guidelines and criteria for teacher certification and recertification. 

• Lists of curriculum standards in mathematics, science, and other disciplines. 

• Newsletters from framework projects to educators in the state. 

• State assessment system overviews and sample items. 

• Position papers from professional associations of mathematics and science 
educators on teaching, learning, professional development, and other topics. 

• In-state evaluations of framework projects. 

• Framework-based professional development materials (including professional 
development models). 

These documents were a rich source of background information on the states and 
their framework projects. In addition to helping members of the study team understand 
the development of the frameworks, the documents we reviewed enabled us to provide 
better support to the expert panel as they assessed the quality of selected framework 
documents. (The methods followed in assessing framework quality are described below.) 

Telephone Interviews 

Telephone interviews were conducted annually with key informants in the states 
and were guided by a semistructured list of interview topics. We wanted to gather 
comparable information from year to year and across framework project states, so the 
same overall topics were explored in each interview. We also wanted to understand the 
individual characteristics of each project, so the topics were sufficiently open-ended to 
capture the full detail of framework projects in different states. 

State-level respondents to the telephone interview always included the framework 
director or the equivalent person within the state department of education or other 
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sponsoring agency (Rutgers University, in the case of the New Jersey project). In cases 
where the math and science framework efforts were headed by different people, we spoke 
with both. Our interviewees included: 

• Project director 

• Evaluator 

• Advisory committee members 

• Participants in the projects 

• Eisenhower state grant coordinator 

• SSI principal investigator 

• State math and science coordinator 

• Other key informants. 

Interviews with the State Curriculum Frameworks Project directors lasted 
approximately 60 minutes; interviews with the other informants ranged from 20 to 40 
minutes. All members of the evaluation team participated in a 1-day training session to 
review the study design and discuss the interview topic guides and case study guides. In 
addition, all evaluation team members received detailed memos before each round of data 
collection, reviewing interview procedures and highlighting issues of importance to the 
particular phase of the study. Although our original data collection plan called for 
interviews with only 10 project directors in Year 4 of the evaluation, all 16 directors were 
interviewed. 

Site Visits 

Using sampling criteria described in Study Design and Data Collection and 
Analysis Plan, we conducted two rounds of site visits. As we indicated in Exhibit B-1, 
we conducted site visits in Delaware, Maine, Massachusetts, Michigan, Florida, 
Louisiana, Nebraska, and Oregon. We visited the latter four states in both years 3 and 4 
of the evaluation. Year 3 site visits consisted of 2 days on-site; year 4 site visits consisted 
of 4 to 5 days on-site. 

The purpose of these site visits was to gather richer data by means of face-to-face 
interviews and to increase the number of interview respondents. Even though we are 
satisfied with the quality of the data collected in the annual telephone interviews, we also 
believe that the additional effort devoted to the case studies enabled us to explore 
common themes in considerable detail and to uncover new ones. For example, our most 
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systematic exposure to the use of frameworks by local educators occurred during case 
study visits to states and school districts in years 3 and 4 of the study. 

The interview protocols used in the case studies were based on the same pool of 
topics and subtopics as the telephone survey interview guide. In fact, the state-level 
instruments were the same. Additional interview protocols were developed for school- 
and district-level interviews. State-level respondents included the following: 

• State Curriculum Frameworks Project director, evaluator, and selected 
committee members. 

• Eisenhower state grant coordinator. 

• Statewide Systemic Initiative (SSI) principal investigator, if any. 

• Chief state school officer or representative. 

• School district officials, administrators, or teachers active in the project at the 
state-level. 

• Representatives of participating institutions of higher education (IHEs). 

The length of the interviews depended on the familiarity of the informant with the 
project. For example, interviews with the project directors typically lasted 2 hours and 
involved interviews at the beginning and end of the site visit. Interviews with the 
Eisenhower state grant coordinator typically lasted only about 30 minutes. 

Interview topics consisted of the following: 

• State context (the education system, educational reform, and the place of the 
State Curriculum Frameworks Project). 

• Planning, development, and implementation of each component of the project. 

• Quality and impact of each component of the project. 

» The State Curriculum Frameworks Projects’ role in the federal strategy. 

Topic guides were designed to allow the interviewer the flexibility to capture the unique 
contextual issues in each state, but were specific enough to see that comparable data were 
gathered across all states. 

Local Case Studies 

Within the 8 case study states, we visited 1 7 school districts. These districts were 
selected because they represented places where at least one of the following conditions 
was present: 
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• State frameworks and/or standards were being implemented. 

• Local educators participated in the development of the state framework and/or 
standards and served as professional development contacts for others. 

• Local curriculum reform or reform orientation was aligned with or exceeded 
state-level initiatives. 

State officials identified these districts for us, in most cases. Our visits to local 
districts lasted 1 day. During our local visits, we interviewed district administrators 
(superintendents and associate superintendents for instruction) and subject area specialists 
in math and science. At schools, we interviewed principals and teachers. Teachers were 
interviewed individually or in groups, depending on their availability at the time of our 
visits. Interviews with teachers were preceded by brief observations of their teaching, 
when possible. All interviews lasted approximately 40 minutes. 

We had two main goals for our district interviews. First, we wanted to understand 
district perceptions of the State Curriculum Frameworks Project, other state reforms, and 
the state context generally. Second, we wanted to gauge how the state framework was 
used and adapted to local circumstances. The local interview questions addressed these 
goals through the following topics: 

• State context. 

• If applicable, local participation in framework planning, development, and 
implementation at the state level. 

• Local perceptions of the quality and impact of each component of the project. 

• Local context and local reform efforts. 

• Local dissemination, implementation, and impact of the framework project. 

At the school level, we wanted to understand how teachers and building 
administrators confronted state and local reforms. We were also interested in what kinds 
of professional development opportunities were available to teachers and whether these 
opportunities were consistent with the frameworks and standards-based reform agendas 
of the state and district. As in the case of district respondents, some building-level 
educators participated in state-level development and implementation of the framework 
projects. We covered the following topics in our school-level interviews: 

• School context. 

• School perceptions of state and district context. 

• Overall vision and goals for the classroom. 







• The state curriculxim framework (the writing process, if applicable; quality; 
exposure to framework-based professional development). 

Collaboration with Other Studies 

The evaluation team was able to leverage data from other related studies conducted 
by SRI and its partners. We purposefully designed the data collection activities to take 
advantage of the national evaluation of NSF’s Statewide Systemic Initiatives (SSI), the 
evaluation of the American Association for the Advancement of Science’s Project 2061, 
the evaluation of the Pew Network for Standards-Based Reform, and the analysis of the 
quality of curriculum frameworks carried out by the Council of Chief State School 
Officers. This coordination resulted in benefits and significant additions to the data 
collected through this contract’s funds. First, our coordination efforts allowed us access 
to internal case study reports on 10 of the State Curriculum Frameworks Projects that 
were located in states that also had SSIs. These case studies were particularly valuable 
because all SSI researchers had been trained to include questions about curriculum 
frameworks in their interviews. The coordination efforts also took advantage of 
overlapping staffing among the studies, allowing some members of the study team to visit 
states (via the SSI evaluation) that were not among our eight-state sample, as well as 
districts (via the Pew evaluation). Our coordination efforts with the CCSSO study 
allowed for the use of an important analysis of a sample of existing frameworks, as well 
as baseline data on frameworks in all 50 states and the District of Columbia. Exhibit B-2 
illustrates which states we were able to visit as a result of other studies. 

Assessing Quality 

To assess the quality of the projects’ curriculum frameworks, we convened a 
distinguished group of experts. A complete list of participants follows this brief 
discussion. The expert reviewers met two times in Washington, DC, to assess the quality 
of the projects’ curriculum frameworks. The first meeting was held in January 1994, and 
the second meeting was held in February 1996. In addition, many of the same group of 
reviewers met in Madison, Wisconsin, in the summer of 1994 and 1996 to assess the 
quality of frameworks from both project and non-project states. The first meeting was 
designed to establish the criteria by which to judge quality. The result was 1 1 elements 
for analyzing curriculum frameworks. Approximately 1 month before the panel 
convened to complete the analysis in each of the subsequent meetings, each member 
received three states’ mathematics or science curriculum frameworks and a set of 
Curriculum Framework Analysis Questions that guided their document review. The 
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analytic questions were derived from the 13 elements the reviewers had established in 
their first meeting in January 1994. Both the analytic questions and the elements are 
included below. 

We encouraged panelists to base their reviews on the analytic questions and to refer 
to the National Science Education Standards, Science for All Americans, Benchmarks for 
Science Literacy, and the National Council of Teachers of Mathematics’ Curriculum and 
Evaluation Standards for School Mathematics. Panelists came to the meeting having 
completed detailed analyses of the three frameworks assigned to their working group. At 
the meeting, four groups of five or six panelists and research team members conducted 
the analysis. Members shared their observations about the frameworks, using the 
project’s analytic questions as a discussion guide. The research staff provided the 
working groups with information about the context within which each framework was 
written, based on information we obtained from extensive interviews with state officials 
in preparation for this meeting. Following a period of discussion, each working group 
drafted a consensus commentary on its assigned frameworks that reflected the views of 
the working group. The research team raised clarifying questions about each review, and 
the reviews were revised one more time. After completing the individual state reviews, 
the reviewers explored cross-cutting themes that emerged from their reviews. 

In addition to assessing quality features of the curriculum frameworks, the 
reviewers summarized what they regarded to be cross-state issues and policy implications 
for the federal government’s continuing support of state systemic reforms in elementary 
and secondary school mathematics and science education. Key ideas and issues were 
further developed on the third day of the meeting by a small group of reviewers and staff. 
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SRI International 

Council of Chief State School Officers 
Policy Studies Associates 
Expert Panel Meeting on Analysis Design for 
State Curriculum Frameworks in Mathematics and Science 

Participants, January 26-28, 1994 
Washington, DC 



Panel Members 

Walter Secada (Panel chair). Mathematics Education, University of Wisconsin-Madison 

Charles Allan, Mathematics Specialist, Michigan Department of Education 

Joan Baron, Student Assessment, Connecticut Department of Education 

Rodger Bybee, Biological Sciences Curriculum Study, and NRC Science Education Standards 

Iris Carl, past president. National Council of Teachers of Mathematics 

Donald Chambers, Mathematics Education, University of Wisconsin-Madison 

Douglas Christenson, Deputy Superintendent, Nebraska Department of Education 

Curtis McKnight, Mathematics, University of Oklahoma 

Celeste Pea, Science Specialist, Louisiana Systemic Initiative 

Senta Raizen, Chemistry, and Director, National Center for Improving Science Education 
Andy Reeves, National Council of Teachers of Mathematics 
Charles Warren, Science Specialist, Ohio Department of Education 

Ex Officio 

Susan Gross, National Science Foundation 
Nancy Loy, U.S. Department of Education 
Charles Stalford, U.S. Department of Education 

Study Staff 

Rolf Blank, CCSSO, Study Director 
Ramsay Selden, CCSSO 
Daniel Humphrey, SRI International 
Patrick Shields, SRI International 
Camille Marder, SRI International 
Andrew Zucker, SRI International 
Bruce Haslam, Policy Studies Associates 
Ellen Pechman, Policy Studies Associates 
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SRI International 

Council of Chief State School Officers 
Policy Studies Associates 

Meeting on State Curriculum Frameworks in Mathematics and Science 

Participants, February 14-16, 1996 
Washington, DC 



Reviewers 

Charles Allan, Mathematics Specialist, Michigan Department of Education 

Douglas Archbald, School of Education, University of Delaware 

Margaret Bondorew, Mathematics Education, Northeastern University, Massachusetts 

Donald Chambers, Mathematics Education, University of Wisconsin-Madison 

Audrey Champagne, Science Education, SUNY-Albany 

Benjamin Dixon, Deputy Commissioner, Connecticut Department of Education 

Charles Doyle-Warren, Science Supervisor, Forest Hills School District, Cincinnati, Ohio 

Lynn Gatto, Science Teacher, Rochester, New York 

Martha Green, Science Specialist, Florida Department of Education 

Susan Gross, Montgomery Co. Schools, MD 

Brenda Hammond, Mathematics Teacher, Montgomery County, Maryland, Schools 

Andrea Keim, Curriculum Frameworks, South Carolina Department of Education 

Curtis McKnight, Mathematics, University of Oklahoma 

Celeste Pea, Science Specialist, Louisiana Systemic Initiative 

Harold Pratt, Biological Sciences Curriculum Study, Colorado 

Helen Quinn, Stanford Linear Accelerator, Palo Alto, California 

Senta Raizen, Director, National Center for Improving Science Education 

Walter Secada, Mathematics Education, University of Wisconsin-Madison 

Ruth Von Blum, Science Education Consultant, California 



Ex Officio 

Annora Dorsey, U.S. Department of Education 
Linda Jones, U.S. Department of Education 
Nancy Loy, U.S. Department of Education 
Larry Suter, National Science Foundation 

Staff 

Rolf Blank, CCSSO 
Daniel Humphrey, SRI International 
Lee Anderson, SRI International 
Barbara Kapinus, CCSSO 
Camille Marder, SRI International 
Ellen Pechman, PSA 
Patrick Shields, SRI International 
Viki Young, SRI International 
Katrina Waiters, PSA 



Guests 

Norm Webb, Consultant, Univ. of WI-Madison 
John Barth, NGA 
Ken Nelson, NEGP 
Alice Gill, AFT 






ELEMENTS FOR ANALYZING STATE CURRICULUM FRAMEWORKS 



At the end of January 1994, an expert group composed of state education staff, 
mathematics and science educators, and researchers worked for three days to identify a 
set of 1 1 elements for analyzing state curriculum frameworks. The elements, outlined as 
follows, represent the panel’s collective position on the characteristics and qualities they 
would look for in a state framework. The elements were the result of panel discussion of 
quality in frameworks, review of current state frameworks, and consideration of national 
standards in mathematics and science. 



Status of the Framework in the State 

• Is the framework a stand-alone document or is it dependent on other 
documents? 

• Is the framework being developed or is it complete? 

Vision of Science and Mathematics Education in the State 

• How is the vision of science and mathematics education presented? 

• What is the rationale for change? 

• What version of the change process is articulated in the document? 

• What are the origins of the ideas behind the vision? 

Function and Intent of the Curriculum Framework 

• What is the intent of the document? Who are the audiences of the 
document? 

• Who (what groups) initiated development of the document? 

• Who was involved in the development of the document? 

• What is the document expected to accomplish? 

Approach of the Framework as a Policy Statement 

• What features of the document are designed to enable teachers and schools 
to make improvements in mathematics and/or science education? 

• What features of the document rely on constraints or restrictions on 
teachers and schools to make improvements in mathematics and/or science 
education? 

• What decisions are left to the local level? District level? School level? 
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• Characterize the nature of the approach of the document (persuasive, 
explicate, exhortative, etc.). 

Conception of the Curriculum 

• How does the document describe the discipline’s importance, 
development, and future? 

• What is the dociunent’s curricular emphasis and explicit instructional 
strategies? 

• How does the document make connections to other disciplines and the real 
world? 

Content of Mathematics and Science Curriculum 

• Does the document reflect national discipline standards? (Which ones?) 

• What are the major categories or topics unique to the state? How are the 
discipline’s boundaries drawn? 

• How is the curriculum organized (strands, themes, big ideas, etc.)? 

• What is the breadth and depth of the subject matter? 

• How is content integrated across disciplines? 

• How are performance standards or expectations communicated in the 
document? 

• How consistent are the principles with the examples in the documents? 

Presentation and Communication of the Content 

• How does the document communicate its content to its audience? (Does it 
use exemplars? Does it rely on other documents?) 

• How readable is the document? Is it appropriate for its audience? 

• Are any factual errors in evidence? 

Pedagogy 

• What conception of learning is presented in the document? 

• What conception of teaching is presented? 

• How are the conceptions of learning and teaching linked and aligned? 

• How are assessment and instruction linked? 

Equity 

• How is equity presented in the document? 

• How are grouping and tracking dealt with? 
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• What attention is paid to the needs of traditionally undeserved, 
underachieving, or special needs groups? 

• To what extent are equity issues infused throughout the dociunent? 

Enabling Conditions 

• How does the dociunent address the school resources and organization 
necessary for improved achievement in mathematics and science education 
(materials, time, professional development, decision-making process)? 

• How does the document address the necessary school district resources 
and community resources? 

• What does the document say about the necessary family resources? 

Policy Connections/Linkages 

• What policy changes does the document suggest are needed? Teacher 
development? Assessment? Books and materials selection? Teacher 
preparation and certification? 

• How is the curriculum framework linked to existing policies on teacher 
development, assessment, materials, teacher preparation, and teacher 
certification? 
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QUESTIONS FOR A QUALITATIVE ANALYSIS OF STATE FRAMEWORKS 

IN MATHEMATICS AND SCIENCE 



Please prepare answers (or notes) on computer disc to the following questions after 
carefully reading the curriculum framework to be analyzed. 



State Context (to be completed by staff) 

1 . Is the framework a stand-alone document or is it dependent on other documents? 

2. Is the framework in draft form or is it complete? 

3. Briefly describe the document. How is the document organized? 

4. Briefly describe the development process. How was the document produced? Who 
was involved in the document’s development? 

5. Briefly describe the state’s efforts to implement the framework. Identify the 
document’s audiences. What is the document expected to accomplish? 

6. How is the framework’s vision statement articulated? What are the origins of the 
ideas behind the vision? What is the rationale for change and how does the state envision 
change occurring? 



Content of Mathematics/Science in the Framework 

1 . Does the framework reflect the national discipline standards? 

2. How does the framework deviate from the main ideas of the national standards? What 
is not included? What is added? 

3. How consistent are the principles for content with the examples (e.g., vignettes, 
instructional strategies, sample problems) in the document? 

4. Does the framework provide performance standards? Are they consistent with the 
national performance standards in mathematics and science? What other standards does 
the document address (assessment standards, teacher preparation, teacher professional 
development, program standards, system standards)? How well aligned are these 
standards with the national standards? 
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5. How are equity issues dealt with in the framework? How well integrated are equity 
issues in the document and in the examples? How are grouping and tracking dealt with? 



Pedagogy and Presentation 

1 . What conceptions of learning, and of teaching, are presented in the framework? Is the 
pedagogy linked to content? Is it linked to the national content standards? 

2. What features of the document are likely to assist teachers and schools make 
improvements in the teaching of mathematics and science? (e.g., readability, depth of 
explanation, use of exemplars, vignettes, sample lessons) 

3. How well does the document make linkages between the disciplines? If the document 
advocates integration of the disciplines, how well does it maintain the rigor and integrity 
of the disciplines? 



Enabling Conditions and Policy Linkages 

1 . Does the framework make a clear linkage between its vision and the policy changes 
necessary to realize the vision? What policy changes does the document suggest have 
already occurred to support the framework’s vision and what additional changes are 
recommended? 

2. How does the document deal with existing state curricular guidance? 

3. How does the document address the necessary school, community, and family 
resources for improved achievement in mathematics and science? 
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State Examples 
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From New Jersey Mathematics Curriculum Framework 
Preliminary Version (January 1995) 



Short-circuiting Trenton 

Ms. Ramirez announces to her seventh grade class that in three 
weeks they will make a journey to Trenton, the capital of New 
Jersey. They will be visiting eight sites - the Capitol, the New 
Jersey Museum, the War Memorial, the Old Graveyard, Trent 
House, the Old Barracks Museum, the Firehouse, and the 
Pedestrian Mall. To ensure that they spend as much time at the 
site as possible, and do as little walking as possible, the class 
must find the most efficient walking tour for the trip, starting and 
ending at the parking lot. 

The first problem that the students must address is finding the 
walking distance between each pair of sites. Ms. Ramirez 
supplies each team with a street map and a ruler; the maps 
identify all the sites to be visited and the routes joining them. She 
assigns each group the task of finding the distances between one 
site and all the others. This turns out to be an interesting task, 
since different groups interpret it differently. Some groups, for 
example, measure the straight line distance between two sites 
forgetting that buildings or ponds might render that walk 
impossible. How to measure the walking distance thus becomes 
an important topic of discussion, as does the question of 
appropriate units. These questions are eventually settled and the 
teachers uses the students’ measurements to write a matrix which 
indicates the walking distance between any two of the eight sites; 
different groups occasionally have obtained different numbers, 
but after discussion, they have arrived at a common answer. 

Ms. Ramirez selects a sample route for the walking tour and 
through discussion with the class explains how the total length 
of the walking tour is obtained from the matrix of information 
that the students generated — you find the distances between 
consecutive sites on the tour, and then add up the walking 
distances along the tour. She now asks her students to work in 
groups to decide on a strategy that they think will produce an 
efficient route (which starts and ends at the parking lot), and to 
assist the group’s recorder in writing a short paragraph 
explaining their strategy. Some groups decide to list all possible 
routes and calculate how long a walk each route entails. (Ms. 
Ramirez asks the students how many possible routes do they 
think they will have to list.) Other groups suggest that the best 
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The students: 



apply mathematical skills 
to solve a real-world 
problem. 



use cooperative group 
work to generate problem- 
solving strategies 



freely exchange ideas and 
participate in discussions 
requiring higher-order 
thinking. 

collect and organize data 
needed to solve the 
problem. 



recognize that there are 
numerous ways to solve the 
problem. 



work in cooperative groups 
to develop alternative 
strategies.. 



The New Jersey Mathematics Standards — 5 1 
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New Jersey Mathematics Curriculum Framework - Preliminary Version (January 1995) 



route is obtained by always going to the nearest site. 

Ms. Ramirez now asks the students to use calculators to carry out 
their strategy and determine the travel time for the routes they will 
be considering. After each group presents its results, the class 
will together compare the various methods that were proposed and 
the accompanying results. Among the questions which 
Ms. Ramirez will ask are; “Do the various methods give the same 
result?”, “Which methods result in a most efficient route?”, 

“What other strategies could we have used?” Responses from the 
students might include; “always use the shortest distance”, 

“never use the longest distance”, “put distances in increasing 
order and use only those that neither make a loop or put a third 
edge into a vertex”, “Which method might we use if we have ten 
attractions to visit?” 



The students: 



compare the variety of 
strategies proposed.. 
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Local Sample 




Fourth Grade 

Performance Standards 

STRAND 1: THE NATURE OF MATTER 

A. Properties of Common Solids, Liquids and Gases 

The student will know that common materials (e.g., water) can be changed from one stare to 
another by heating and cooling. 

STRAND 2: ENERGY 

A. Energy Exists in Many Forms 

The student will: 

1. Recognize various forms of energy (e.g., heat, light and electricity). 

2. Recognize how to trace the flow of energy in a system (e.g., as in an ecosystem) 

B. Objects Store Energy 

The student will identify stored energy sources. 

C. Uses of Electrical Energy 

The student will recognize different ways in which electrical energy is used. 

STRAND 3: FORCE AND MOTION 

A. Simple Machines Apphcarions 

The student will: 

1. Recognize forces of gravity, magnetism and electricity operate simple machines. 

2. Experiment with forces and communicate the effects of forces on objects. 

B. Effects of Mass on Motion 

The student will demonstrate that the more massive an object is, the less effect a given force 
has on that object. 

SXRAND 4; PROCESSES THAT SHAPE EARTH 

A. Features of Earth’s Surface 

The student will: 

1. Recognize that 75 % of the surface of the Earth is covered by water. 

2. Commuiticate that the Earth has layers composed of various materials. 

3. Describe the processes involved in the movement of the Earth’s crust 

4. Investigate the effects of water, sun and atmosphere on the Earth’s surface. 

B. Water Cycle 

The student will: 

1. Explain physical properties of water. 

2. Identify how the amount and type of water in an environment affect the adaptations of 
living things. 
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STRAND 5; EARTH IN SPACE 

A. Components of Solar System 

The student will: 

1. Describe and compare the components of the solar system. 

2. Recognize that the planets differ in size, characteristics, and composition and that they orbit 
the sun in our solar system. 

B. Sun as a Star 

The student will: 

1. State the arrangement of planets in our solar system. 

2. Recognize that, in addition to the sun, there are many other stars that are still farther away. 

STRAND 6 PROCESSES OF LIFE 

A. Cycles of Life 

The student will observe the various stages of development of different organisms. 

B. Human Body System Identification 

The student will: 

1. Recognize that the human body is made of systems with functions that are related. 

2. Explain how bones and muscles work together to allow for body movement 

3. Explain how foods help the body. 

4. Explain how foods help the body. 

C. Living Things are Made of Cells 

The student will recognize that similar cells form different structures. 

STRAND 7 HOW LIVING THINGS INTERACT WITH THEIR E NVIRONMENT 

A. Ecosystems 

The student will: 

1. Describe how the earth’s air, water and soil support varied life forms. 

2. Recognize that living things compete in a climatic region with other living things and 
that structural adaptations m^e them fit for an environmenL 

3. Explore and discuss the characteristics of a diverse ecosystem (such as a rain forest) 

4. Describe characteristics that enable plants and animals to survive in various habitats. 

B. Interactions Among Plants and Animals 

The smdent will: 

1. Recognize that animals eat plants or other animals to acquire the energy needed for survival. 

2. Demonstrate the enersy flow (food web, energy pyramid) in each ecosystem. 




STRAND 8 NATURE OF SriENCF. 

A. Observe 





The student will utilize appropriate observation skills in science activities. 

B. Classify 

The student will develop and use various scientific classification systems. 

C. Measure 

The student will; 

1. Utilize the SI measurement 

2. Choose the appropriate unit for measurement for an activity. 

D. Collect/Record Data 

The student will; 

1. Collect scientific data from activities. 

2. Record data in an appropriate format 

E. Question 

The student will formulate appropriate questions based on activities. 

F. Identify Variables 

The smdent will identify variables within a scientific activity. 

G. Interpret Data 

The student will interpret data collected through scientific experimentation. 

H. Predict 

The smdent will use prior knowledge and collected data to determine the outcome of an 
activity. 

I. Draw Conclusions 

The smdent will draw conclusions based upon collected data 

J. Follow Safety Procedures 

The smdent will apply proper safety procedures to scientific activities. 

K. Use Science Equipment 

The smdent select and use expropriate equipment for scientific activities. 
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Moving Away From Earth 

Tlie solar system 
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Fourth Grade Science 

Sample Graphic Organizer 
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Third Rock From The Sun 







Journey to the Center of the Girth 



_STATE STANDARD^ SC.C.1.2.1, SC.G.1.2.5 



■~1 3.1 3.2 3.4 3.7 3.8 



PERFORMANCE STANDARDS I 
PURPOSE i 



6B1-3, 7B1, 8A-B, 8G, 8J8 



To recognize that the human body is made up of systems with functions that are related. 

To explain how bones and muscles work together to allow for body movement 
To recognize that animals eat plants or other animals to acquire energy needed for survival. 
To recognize that similar cells form different structures. 

To explain how food helps the body. 



materialTI 



Butcher paper, art supplies, macaroni, skeleton model, microscope, construction paper 



STRATEGIES 



• Use a microscope to examine various types of cells. 

• Draw and label the different types of cells examined. 

• Brainstorm a list of energy sources for the body. 

• Create a “Food Helps The Body” book. 

• Cooj^rative groups trace and cut out body. Each group will “build” a selected/assigned system and 
glue it to the body shape. Groups will label system components and report to the class on system 
function and how the system interacts with other systems. 

• Play a game matching body organs to the system they belong to. 

• Use a skeleton model to show how bones and muscle work together. 

• Create a skeleton using macaroni for bones and red yam for muscles. Glue on construction paper. 

I ASSESSMENT I 

Student participation in activities 
Student products rubric 



RESOURCESl 



Moving Heavy Things by Jan Atkin (Houghton Mifflin) 
The Magic School Bus by Joanna Cole, (Scholastic) 

I CONNECTIONS 

Art (body model) 
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Moving Awa\ From Earth 



I STATE STANDARDS | SC.E.1.2.4, SC.E.1.2.5, SC.E.2.2.1 

g 3.2 3.3 3.4 3.6 3.8 

PERFORMANCE STANDARDS | 5A1-2, 3B, 5B1-2, 8A, 8B, 8E, 8G 



PURPOSE g 






To describe and compare the components of the solar system. 

To state the arrangement of planets in our solar system. 

To recognize that the planets differ in size, characteristics, and composition and that they orbit around 
the sun. 

To recognize that in addition to the sun, there are many other stars that are still farther away. 



I materialT"! 

Art supplies, measuring devices, graph paper, science journals 
I STRATEGIES 

• Take a field trip to B.C. C. Planetarium. Reflect on experiences in journal. 

• Arrange planets in order across the length of a hallway or classroom. Discuss relative size an distance 
and why a true scale model is difficult to build. 

• Desing and participate in a body movement activity that simulates planets rotating and revolving 
around the sun. 

• Graph planets according to a selected characteristic. 






ASSESSMENT j 



Teacher observation 

Student participation in activities 

Rubric 



resourceT~1i 



CONNECTIONS g 



ERIC 




Discover The Wonder, Module A (Scott Foresman Science) 

The Magic School Bus , “Lost in The Solar System" , by Joanna Cole (Scholastic) 
Voyager: An Adventure Through Space, by John Gust^son (Scholastic) 



Art (model solar system). Math (measurement, graphing). Language Arts (journal writing) 







A Living Planet 



I STATE STANDARDSl SCD. 1.2.4 

r ~l 3.2 3.3 3.4 3.8 

PERFORMANCE STANDARDS i lA, 4A2-4, 8A-E, 8G-K 
PURPOSE i 

• To communicate that the Earth has layers composed of various materials. 

• To describe the processes involved in the movement of the Earth’s crust 

• To investigate the effects of water, sun, and atmosphere on the Earth’s surface. 

materialTI 



Art supplies, ice cream, ice cream cones 

I STRATEGB^ 

• Cooperative groups will research and design a demonstration of the effects of water, sun, or 
atmosphere on the Earth’s surface. 

• Create layers of the Earth out of different colors of ice cream and ice cream cones. Discuss layers 
of earth as you eat the ice cream. 

• Design a demonstration of the movement of the Earth’s crust. 

ASSESSMENT § 



Rubric 

RESOURCWl 

Discover the Wonder, Module B, (Scott Foresman Science) 

The Magic School Bus : Inside the Earth by Joanna Cole, (Scholastic) 

CONNECTIONS g 

Art (paper mache). Math (graphing, measuring). Language Arts (writing)) 
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Ma\ The Force Be With You 

.^^^^_^TAT^^STANDARD^ SC.c. 2.2.1, SC.C.2.2.3 
g 3.2 3.3 3.7 3.9 

I PERFORMANCE STANDARDS \ 5A1, 2B, 2C, 3A1-2, 3B 

PURPOSE 1 



• To recognize various forms of energy and identify stored sources of energy. 

• To recognize different ways in which electrical energy is used. 

• To recognize that forces of gravity, magnetism, and electricity operate simple machines. Experiment 
with forces on objects. 

• To demonstrate that the more massive an object is, the less effect a given force has on that object. 

I MATERIAL^ 





Art supplies, chart paper, booklet, simple machines 

STRATEGIES 



• Brainstorm a list of electrical energy and other energy uses at home and school. 

• Create a booklet describing ways to reduce the amount of electrical energy used at home and school. 

• Create a graph of stored energy sources. 

• Set up experiments using gravity, magnetism, Md electricity on simple machines. Record results. 

• Set up experiments to show that the more massive an object is, the less effect a given force has on 
that object. 
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Third Rock From The Sun 




The Indian River Lagoon 




^ SC.F.1.2.3, SC.G.1.2.2 

I I 3.1 3.2 3.4 3.5 3.8 



STATE STANDARDS 



SC.B.1.2.1, SC.D.1.2.2, SC.D.1.2.3, SC.F.1.2.2, 
SC.F.1.2.3, SC.G.1.2.2 



3.1 3.2 3.4 3.5 3.8 



PERFORMANCE STANDARDS"! 4A 1 , 4B 1 -2, 6A, 7 A 1 -4, 7B2, 8 A-B, 8D-E, 8G, 81 




° To observe various stages of development of different organisms. 

° To describe how air, water, and soil support varied life forms. 

° To describe the characteristics of a diverse ecosystem and demonstrate the energy flow. 

° To recognize that living things make structural adaptations. 

° To describe characteristics that enable plants and animals to survive in various habitats. 

* To identify how the amount and type of water in an environment affect the adaptations of living 
things 

° To explain the properties of water and recognize that 75% of the Earth’s surface is covered with 
water. 

MATERIAL^ 

Art supplies, science journals, chart paper, lagoon habitat components 
I STRATEGIES! 



o Take regular field trips to the Indian River Lagoon. Record observations in science journal. 

® Make a chart of organisms found in the air, water, and soil at the lagoon. 

® Discuss the habitats that support the plants and animals in the lagoon. 

° Design, set up, and maintain a lagoon habitat in the classroom, including plants and animals. 
Observe and record daily observations. 

® Observe organisms for competition with other living things and for structural adaptations. 

® Invite a guest speaker to discuss the lagoon and create a mural of lagoon habitats. 

® Create a paper mache globe showing that 75% of the Earth’s surface is covered with water. 



uiving into 2>cience oy h'eggy k. Perdue ( A Good Year Book) 

The Indian River Lagoon: An Exceptional Lagoon, Florida Dept, of Natural Resources 
Protecting Paradise: 300 Ways to Protect Florida’s Environment by P Cavanaugh & M. Spontak 
(Phoenix Publishing) 

A Walk In The Wild : Exploring a Wildlife Refuge by L. Ward and L. Jacques (Charlesbridge) 
CONNECTIONS j 
Art (mural). Language Arts (journal) 




Student products. 




Pages? 
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NOTICE 

REPROmiCTION BASIS 




This document is covered by a signed “Reproduction Release 
(Blanket)” form (on file within the ERIC system), encompassing all 
or classes of documents from its source organization and, therefore, 
does not require a “Specific Document” Release form. 




This document is Federally-funded, or carries its own permission to 
reproduce, or is otherwise in the public domain and, therefore, may 
be reproduced by ERIC without a signed Reproduction Release 
form (either “Specific Document” or “Blanket”). 




